Managing Big Data in Clusters and Cloud Storage

I played the role of a data analyst for the company to create a table with data describing an underground tunneling project.

Each of these TBMs will generate a large volume of data as it operates. Each TBM will generate the data slightly differently. Simulated versions of the three TBM-generated datasets are provided. You must create a table on the VM and load these datasets into it. Then you must create and upload a document describing the steps you performed to complete this task.

After exploring the files containing the data, I observed that there are three text files present.

You can review the text files using the shell commands below

Using HUE (Hadoop User Environment) on a virtual Machine, I used Impala to create a database dig

The next step I took is to load the tables into a table each using the following Impala codes

Presence of Null Values

The Tables have null values so I have to write a query so that impala would recognize them as null values

Union of Tables

I used the SQL code below;

After performing the steps described above, I ran the following queries and they produced the following result sets:

The results

The results

Thanks for reading :)

Related posts:

On my journey to a life of crypto passive income, I was lucky to meet some people, decent people, very hard to find when there is money involved. One of those people is Craig, he started his project…

With the way the world is changing, we can predict AI domination in the coming years. With the world of marketing changing every day, technology is also changing and content creation itself is…

How many hours do you spend on your phone daily? The answer would probably be in some hours. Be it one hour or eight hour, imagine spending it in pure quality. We all have become slave to the…