Implementing a large scale social simulation using the New Zealand BeSTGRID computer network: A case study

The Modelling Social Change (MoSC) project is a large scale social simulation project which investigates changes in the social structure of New Zealand by simulating inter-ethnic cohabitation patterns using models populated with unit-level census data. This paper is presented as a case study of the implementation of this simulation project on a high power cluster within the New Zealand BeSTGRID (www.bestgrid.org) computer network by a researcher with a social science (rather than computer science) background. The aims of thisxD;paper are two-fold. Firstly, it aims to encourage other researchers to investigate the potential of implementing their own research in a grid based environment by highlighting the benefits that were gained from using grid computing. Secondly, it aims to inform the managers and systems administrators of gridbased systems of the some of the difficulties which a new user may face and how they may be of more assistance in introducing new researchers to the technology. The MoSC research project applies computer based social simulation techniques to cohabitation and demographic data from the New Zealand Census in order to test models of New Zealand's social structure in the rapidly changing demographic and economic conditions of the period 1981 to 2006. The central research question of the project focuses on whether the social structure of partnerships in New Zealand, as reflected in the distribution of inter-ethnic marriages and the choice of cohabitation partners, has became more highly stratified and segregated over this period. In addition it also examines what factors and/or social processes have lead to the variability (or lack of variability) in the distribution of these inter-ethnic partnerships through the sensitivity testing of the simulation model and the use of 'feedback loops' to model recursive social processes. The simulation was written in Java and run on the Auckland cluster of the BeSTGRID computer network; a system with 80 CPUs, 160 Gigabytes of memory and 2500 Gigabytes of hard drive space. The processing power of the cluster allowed the simulation to be run at a city level, with unit data that provided demographic information for all of the single eighteen to thirty year olds listed in the census in the Auckland, Wellington and Canterbury regions. The use of such a large population dataset represents a huge advance over most social simulation experiments that tend to be run using small samples of data. The cluster also allowed for the parallel processing of code which provided the opportunity to run an evolutionary optimisation algorithm across the parameter space in order to find optimal combinations of the partnering parameters. The implementation of the simulation on the grid wasn't completely trouble-free. Being a 'guinea pig' (a new user with a project that was very different from the other users) created some frustration as many of the systems and processes for using the grid were not set up with social science users in mind. Stepping up from basic Java programming on a single machine to parallel processing on a complex system was a steep learning curve and increased the reliance of the project on the technical staff who look after the cluster. Overall, the application of a large scale social simulation to grid technology has been a positive one. It has allowed the creation of simulation models on a much larger scale than are typically seen in the area of social simulation and has opened the door for future social science research using grid technology.