The rapid development of technology and the introduction of Government 2.0 (Gov 2.0) generate large data sets. At the same time research into these large data set has become popular and has been coined 'Big Data'. On the other hand there is a long standing discipline for controlling and governing the important data of an organisation called Electronic Records Management (ERM). This paper investigates the challenges identified in the literature for Gov 2.0, Big Data and ERM. We conclude that a key building block in providing the right information to the right stakeholders could potentially be ERM with its well established governance policies. Future research is necessary to address the specific constraints and expectations placed on governments in terms of data retention and use.