Big data itself, although only known in the last few years. The history of big data begins in the early 60s – 70s. It was in those years that ‘citizens of the world’ began to be literate in data and its analysis through statistical science.
Meanwhile, the emergence of Facebook and various other social media in the 2000s made people begin to realize how important user data is that these social media platforms have. However, problems arose, because the existing input from social media platforms at that time was too large to be stored and processed. This issue was then slowly addressed, starting with the emergence of Apache Hadoop (currently with Apache Spark) and NoSQL.
Spark itself is an open-source framework that was created specifically for storing and analyzing big data. With Spark, problems related to big data can be handled. Since then, the volume of big data has grown very rapidly.
Furthermore, the involvement of technology in every aspect of human life makes big data input bigger. The more information a machine can get about its users.
How Big Data Works?
You who are an entrepreneur can use big data to run your company. With this big data, you can get a lot of insight to do company planning. So that’s why you also have to understand how to treat big data.
Compared to traditional data that can be handled by “ETL” aka extract transform and load, big data processing is much more complicated than that.
The reason is because big data consists of a set of different inputs and must be processed before it can actually be processed. For example, you can collect data in the form of consumer comments in public spaces, data on consumer photos with your products on social media, or even traffic on your website and social media pages.
After processing and creating a uniform format, then you can submit the data to your business analyst.
To be able to manage data properly, the most important factor that you can’t forget is how you store the data. The choices are indeed very many, but for those of you who have limited funds, of course cloud storage is one solution.
Your large investment in collecting and managing data can only be paid off if you can perform big data analysis.
Unfortunately, as we explained earlier, traditional data processing software will not be able to process big and complex big data. You must be able to model data using machine learning and artificial intelligence so that it can be analyzed properly.
When the data has been analyzed, believe me you will find a lot of input, insight, and new discoveries that you can use for the progress of your company.
Big Data Concrete Application
With the increasing size of big data, and big data processing capabilities are also increasingly sophisticated. The current application of big data is no longer a figment. Big data applications that were originally aimed at companies for marketing and maintenance purposes are now starting to be used in many fields.
Big data is so popular that it has even attracted big companies like IBM, Microsoft, SAP, HP, Dell, and Oracle Corporation to invest more than US$ 15 billion in the development of specific software for big data management and analysis.
In 2010 alone it was discovered that the big data analysis and management industry was worth up to US$ 100 billion worldwide with a development value of up to 10% per year. This value beats the value of software business development by up to 2 times.
So what fields are currently using big data technology concretely?
You could say the use of big data in the government sector makes the government process much easier and more efficient. However, with the mastery of big data, it means that the government has stronger control and control over its people.
One of the uses of big data in the government sector that is quite popular today is the CRVS system (Civil Registration and Vital Statistics) which was introduced by WHO as a civil registration which includes data on births, deaths, in detail including causes of death, and history of marriage and divorce.
This system has been popular for a long time. The problem is, in this traditional way the accuracy of the data is in question. Big data is the answer!
Several countries that have initiated big data-based CVRS include Rwanda, Oman, and New Zealand. There is also the WHO with its Monitoring of Vital Events (MOVE-IT) system and African countries with their iCivil Africa.
Big data analytics enables healthcare providers to improve overall services. This includes personalized medicine, prescribing analysis, clinical treatment risks, predictive health analysis, and much more.
Moreover, in the world of health data collection is not new. Many health services have utilized electronic medical record technology that contains digital health information.
With the existence of electronic medical records for a long time, today’s big data management and analysis technology can help health care providers perform analysis of existing collections of electronic medical records.
The only problems with big data analysis in healthcare are ethical and privacy issues. Because big data collected in the world of health is generally patient health data, which in any country must be protected by law.
Well, I hope that this last problem can be handled, because if big data analysis is actually implemented 100% in the health sector, the benefits can be very large!
It is not a concrete implementation of big data, but the discovery of big data at this time has made many educational institutions realize that the previously existing data processing science was not enough to deal with this big data problem.
Currently, many universities in the world, especially in the United States, even create master’s programs aimed at producing masters in the field of big data processing and analysis.
In addition, there are also several short programs in the form of private boot camps such as The Data Incubator or General Assembly which specifically have the aim of producing experts in the field of big data processing and analysis.
Furthermore, this issue is also raised in business schools around the world. Current business schools should be aware that a business school graduate must be able to handle big data which is no longer one-size-fits-all like traditional data processing.
So that in the end they can produce business consultants and marketing experts who are able to process big data as a whole.
Talking about the use of big data, of course, it doesn’t feel good if we don’t discuss its use in the media world. Because today’s social media is actually a collection of big data that is ready to be collected and analyzed by media practitioners.
In the past, these practitioners used to collect information from newspapers, magazines, and television programs separately. So currently the existence of social media and the internet makes practitioners have to collect millions of data related to consumer information to consumer behavior in it.
The goal? Of course targeted ads and content!
If you are still confused, when you open a website page, you see ads that seem to suit your needs. This is the result of brilliant big data processing. For those of you who have an interest in guitar, for example, you must often search for guitars on Google every day. In fact, you may even have been recorded as going to guitar shops on Google Maps a few times.
That’s where Google then gets ‘input’ that you (the user) are guitar fans, who may be looking for the latest guitar. Then Google will issue output in the form of guitar ads that often appear when you open a website page. The goal? Of course, so that the ad reaches those of you who are really interested in the guitar, so that you click on the ad, and maybe even buy a guitar from the ad.