Big data refers to datasets that are terabytes to petabytes (and even exabytes) in size, and the massive sizes of these datasets extend beyond the ability of average database software tools to capture, store, manage, and analyse them effectively. Big data is not just about storage or access to data its solutions aim to analyse data in order to make sense of them and exploit their value. In this context, big data is a popular phenomenon that aims to provide an alternative to traditional solutions based on databases and data analysis. The problem of managing and extracting useful knowledge from these data sources is currently one of the most popular topics in computing research. The classic methods, algorithms, frameworks, and tools for data management have become both inadequate for processing this amount of data and unable to offer effective solutions for managing the data growth. These data come from everywhere: sensors used to gather climate, traffic and flight information, posts to social media sites (Twitter and Facebook are popular examples), digital pictures and videos (YouTube users upload 72 hours of new video content per minute ), transaction records, and cell phone GPS signals, to name a few. In this paper, we present a revision of the new methodologies that is designed to allow for efficient data mining and information fusion from social media and of the new applications and frameworks that are currently appearing under the “umbrella” of the social networks, social media and big data paradigms.ĭata volume and the multitude of sources have experienced exponential growth, creating new technical and application challenges data generation has been estimated at 2.5 Exabytes (1 Exabyte = 1.000.000 Terabytes) of data per day. These new challenges are focused mainly on problems such as data processing, data storage, data representation, and how data can be used for pattern mining, analysing user behaviours, and visualizing and tracking data, among others. The combination of big data technologies and traditional machine learning algorithms has generated new and interesting challenges in other areas as social media and social networks.
A number of libraries such as Mahout and SparkMLib have been designed to develop new efficient applications based on machine learning algorithms. The rise of different big data frameworks such as Apache Hadoop and, more recently, Spark, for massive data processing based on the MapReduce paradigm has allowed for the efficient utilisation of data mining methods and machine learning algorithms in different domains. Big data has become an important issue for a large number of research areas such as data mining, machine learning, computational intelligence, information fusion, the semantic Web, and social networks.
0 kommentar(er)
