Basic Concepts of Big Data and Artificial Intelligence in Climate Data Management

In the context of the 4th Industrial Revolution, several important technological developments for observed are big data, artificial intelligence (AI), machine learning, block chain, and financial technology. Some of this technology is not new technology, but existing technology since a long time ago. However, in recent years, these four technologies have experienced. Very rapid development brings unimaginable socio-economic impacts previously. Experts state that these four technologies will form the basis for emergence of advanced technological breakthroughs.

Big Data and Artificial Intelligence are two things that have emerged in industry 4.0.

Big Data and Artificial Intelligence have been widely applied in many ways, including climate data management with the aim of producing weather and climate information and predictions. Currently, big data in Asia Pacific is mostly used for data monetization. A variety of platforms there are and various big data activists use big data to provide prediction services and input for business.

However, the regulations regarding governance in many countries are not yet comprehensive data (such as how data retrieval and processing should be carried out, guidance for the cloud services, data audit processes, and data ownership issues) are recognized as obstacles and confusing big data activists in Indonesia. To date, the government has not issued specific initiatives undertaken to regulate the governance of the data mechanisms required by big data activists.

Understanding Big Data and Artificial Intelligence

Big Data is a very large data set that has certain characteristics that cannot be processed using a single conventional computer. The data is very large and continues to grow. Big Data has 4 characteristics known as 4 V, namely Volume (big data), Velocity (speed), Variety (type of data) and Veracity (data quality).

In another reference there are additional 3Vs of Big Data covering Value, Variability and Visualization.

The data included in Big Data is not only structured data but also unstructured data. The differences are as follows:

• Structured data is data that is arranged in one place with a clear schema so that it is easy to analyze and integrate with other structured data, including data in databases or spreadsheets.
• Unstructured data is data that is available in various forms that are not easily classified. Examples of unstructured data such as photos, graphic images, streaming instrument data, webpages, pdfs, PowerPoint presentations, blog content and so on.

Data warehouse at the National Computational Infrastructure (NCI) was managed by the Australian National University (ANU). NCI claims to be the institution with the fastest supercomputer in the south of the earth, having data storage reaching 100 petabytes.

Artificial Intelligence transferring the human brain to a machine

Artificial Intelligence is a man-made technology that combines mathematical abilities with algorithmic statistical processes. Some people call Artificial Intelligence like transferring the human brain to a machine so that it can think like humans.

Artificial Intelligence makes a machine with artificial intelligence to make decisions automatically which are usually done by humans.

Decisions made by machines with Artificial Intelligence technology in them are based on previous data or information that is recorded into knowledge stored in the machine's database.

Big Data and Artificial Intelligence in Climate Data Management

Climate data that has been collected for decades from various places with various types of data is Big Data. Climate data as Big Data consists of text in the form of row data, radar and satellite images to video animations such as the movement of tropical cyclones.

This climate big data will require high performance computing (HPC) to process it into a new climate information without adding it to a database.

Climate data in the Big Data concept which is then processed with Artificial Intelligence does not only refer to the climate data itself.

Metadata on climate data such as observation coordinates, photos of observation tools, types of tools used, records of equipment damage and others will be stored as part of Big Data. The characteristics of climate data as Big Data are as follows:

Volume (data capacity)

Volume in the Big Data concept refers to the amount of data generated at any time. We can imagine the volume or amount of climate data that has been observed over the years from various places around the world.

The climate data is generated from manual observations as well as automatic observations that record data for up to a matter of minutes and of course will continue to grow.

Velocity (speed)

Velocity is another big data feature. It takes a speed in data transfer and compiling very large climate data to be accessed simultaneously. For example, we want to access climate data for one year from all climate observation stations, at the same time we process sea surface temperature data spatially and analyze satellite images in real time.

Variety (type of data)

Variety in Big Data relates to the type of data. If in the past climate data was only structured text data, now climate data also includes graphic data such as radar images and satellite images and videos that turn climate data into unstructured data.

Veracity (data quality)

Veracity in the big data concept is concerned with data quality. In the process of collecting climate data, there may be errors in data typing or data recording errors due to interference with climate observation equipment.

Then what is the position of Artificial Intelligence in terms of managing climate data or information? Climatologists have long studied the correlation of climate data with the factors that play a role in the climate system. For example, the relationship between El Nio and global climate patterns.

Artificial Intelligence and climate data analysis

Artificial Intelligence will be a supporting technology to get climate data analysis with the existing climate system.

Although weather and climate are natural phenomena related to the laws of physics, with Artificial Intelligence algorithms we will be able to make weather and climate predictions. In Artificial Intelligence there are approximately three methods to support Artificial Intelligence as an artificial intelligence, namely:

Fuzzy Logic(FL)

The methods used by machines to adapt or imitate the way living things adjust to conditions then give a non-rigid decision that is just like 0 or 1.

Evolutionary Computing (EC)

An approach that uses an evolutionary schema with large amounts of individual data. The results of the approach then provide a test to select the best individuals to generate the next generation. Its application, for example, uses the idea of ​​mutation and crossbreeding.

Machine Learning (ML) or machine learning

This method is the most widely developed technique today. This is because Machine Learning is widely used to replace or imitate human behavior and then make certain decisions. Artificial Intelligence technology will be able to read climate system patterns based on previously stored climate data to then generate weather or climate predictions based on these patterns.

With Artificial Intelligence technology connected to climate Big Data, learning from Artificial Intelligence technology on climate patterns will be better at analyzing climate patterns which then produce more accurate climate predictions. Thus a review of the Basic Concepts of Big Data and Artificial Intelligence in Climate Data Management, may be useful.