Can you hack data?

This article is a re-post from an original post written by Aishwarya Sahu, who is a junior engineer at Aquamonix. The article was published on her Linkedin blog and we liked it so much that we wanted to share it with you.
Especially good read if you are/were hesitant about joining hackathon yourself or were simply wondering how a ‘data hackathon’ is different to any other. Here is the first hand account without further ado!

Can you hack data?

You absolutely can!

Here’s how hacking data changed my perception of hackathons.

I ventured into my very first hackathon by participating in Datadriven.SG Hackathon. I knew how hackathons work because of the many stories I have heard from my friends, but I was excited to experience it first-hand.

Unlike most hackathon, I knew this was different. The focus of what we make was open ended. The solution could be anything to any problem. The only thing that was definitive was that we use data to understand and thereby solve the problem we choose.

 

The folks from Keboola planned the Hackathon such that we got an understanding of the tools to which we were given access. Heck, that wasn’t even a restriction, we could use any tools as we liked, but to use a new one is always exhilarating. So to make it easier on us, there were intro sessions during which we were briefly introduced to the tools and technology that were at our disposal. These were Tableau and Yellowfin.

While many were familiar with Tableau and played by their strength of familiarity, others went on to using Yellowfin to explore the new terrains. Turns out, our team was the only one that used both. Why? Well, our key motivation was to learn as much as we can. So we tried to acquaint ourselves with all the tools at our disposal. But most importantly, we wanted to tell our story well.

I walked into the hackathon without really knowing a lot of people. Didn’t know who I would meet or where I would fit in. After the introductions, I was approached by someone who would then be a part of our team. We both bumped into two more people, and just before lunch, bumped into the last member of our team. We called ourselves FourPlusOne – because four females and one male. Very unusual for the overall gender ratio we had at the hackathon, but that didn’t matter because we worked well with each other. For a team with relatively less experience with data analysis, our strength lied in the fact that we listened to one other but also because we all loved Microsoft Excel. We brainstormed and picked each other brains to no end. Finally, we had a story to which we all related.

 

One of the biggest dataset that was given to us was from Skyscanner of the travel bookings made to and from Australia. We began brainstorming by thinking of ways to make the existing travel search more resourceful, either for the company or the user. After jumping around from different ideas, we all realized that our individual ideas can be brought under one common solution – to reduce travel related stress for the customers. Thereafter, we worked on getting more data to work on our respective chunks before putting it together.

Day two started with us straight up playing with the data. We started with the most brutal process – data cleaning. We cleaned it by different constraints, completeness and consistency. Our next hurdle was to effectively visualize the data. We first started off by using Yellowfin, and we were stuck. Fortunately, we had plenty of tech support for all of the tools we used. After using Yellowfin we came on a crossroad whether we continue using Yellowfin or switch to Tableau. We figured why not both? So that’s what we did. In the end that really helped us out as these two tools complemented each other well.

So how did we plan to reduce stress for travelers? But using different metrics to measure stress, such as offering a more comprehensive understanding of price rates by the month, arrival time so they passengers have a better understanding of any delay history for a carrier, data for how crowded a certain airport may be by the month and amenities at the airport so that the passengers can plan their trip accordingly. We want this tool to be useful for all types of passengers, whether it’s a businessman or a backpacker, everyone should benefit from this.

 

And so it was time, all the teams were to pitch their ideas to the jury. As we were running out of time, we rushed through the presentation preparation and we definitely did not feel up to the mark. Mostly because we felt our presentation wasn’t representing our ideas well. But time was up and we had to roll with it.

Our presentation was almost at the end, so we team members bonded over our jittery feelings. We listened to the other pitches and were absolutely loving some of the simple but brilliant solutions to various problems. Then it was our turn and the idea was pitched. Six minutes, and it was all done. On the other side of the pitch, we started feeling a little good about the work and effort that went into our solution. We were content that we saw it through and didn’t quit.

After all the pitches, the jury went away to decide who were the three winning teams. There was excitement and anticipation in the air. But most importantly, there was a sense of pride and joy everyone had for finishing the hackathon. The jury were back and the results were announced. I don’t know if you’ve guess it, but we won! The whole team was in absolute shock when we heard the result. Our souls were on cloud nine but our bodies, frozen to the chair, remained unshaken until we were called up on the stage by the jury. It was absolutely unreal! For what mattered though, the jury loved our idea and it didn’t matter whether the actual visualization was complete. We had enough to communicate our notion well with them.

 

So that was my first hackathon experience, all golden thanks to the support from Keboola, Yellowfin, Tableau, Minerra and Billigence. Heaps of coffee kept us all going, which also led us all to learn to use an Espresso machine, which is super neat! The good food delighted my soul and the great company of all the participants added some amazing insights to my life. But the cream of the crop was definitely being a part of the winning team of Datadriven.SG Hackathon!

 

Did you like it?

Let us know so we can improve

+3 Vote DownVote Up