Hackathon Uses Machine Learning to Improve Cybersecurity

Machine learning (ML) and artificial intelligence (AI) are hot technologies with significant promise for shaping the future, but it isn’t enough to understand ML or AI theory. Sure, you can gain a lot of knowledge and information from reading and studying, but there are some things you just have to do to really understand. The best way to learn is to do—and that’s where Kaggle comes in.

Kaggle.com

Kaggle—which you can find at Kaggle.com—is a platform for students and practitioners of machine learning where they can participate in competitions and class assignments, communicate with other team members and teachers, and get hands on experience. Kaggle gives would-be data scientists access to an environment and data sets where they can test and explore solutions.

At the same time, Kaggle also provides an opportunity for crowdsourcing creative and innovative solutions to existing problems. By setting up competitions around real-world scenarios, Kaggle can help develop machine learning models to address real-world challenges while also providing an engaging and educational experience for the participants.

Kaggle was recently acquired by Google, and offers pretty much the only online hackathon platform that is free for small scale competitions.

Solving Problems through Hackathons

One recent Kaggle hackathon competition focused on finding a solution for identifying insincere questions on Quora. Quora is a platform designed to allow people to learn from each other. People can pose questions, and the Quora community will answer. As with virtually any online platform, though, Quora must also deal with trolls and users who abuse the platform. Quora needs to have an effective method for blocking or weeding out insincere questions—questions that are based on false premises or poor assumptions, or questions posted to the platform just to make a statement or gain attention. These sorts of questions impact the value and credibility of the platform as a whole.

To solve the problem, Quora turned to Kaggle. Quora set up a hackathon competition asking Kaggle members to develop machine learning models capable of identifying and flagging insincere questions. Quora hoped the Kaggle community could help them develop more scalable methods to detect toxic or intentionally misleading content.

Determining Malicious Intent

When it comes to cybersecurity, there are certainly things that are black and white—in other words, things that are obviously acceptable and things that are absolutely malware or exploits. The vast majority of network traffic and actions by users, however, fall into a nuanced gray area where determining intent is more difficult. This where Wallarm applies its AI engine and that’s also why Wallarm has turned to Kaggle to work on innovative solutions for identifying malicious intent.

Wallarm is sponsoring a hackathon on Kaggle to work together with the community to find better detection solutions through innovative ML models. The traditional approach to detecting malware or determining malicious intent relies on sequentially comparing incoming requests against a database of predefined patterns or signatures. There are multiple problems with this approach. Developing signatures is reactive, classifying inputs with signatures is frequently inaccurate, and the sheer volume of traffic can quickly overwhelm traditional security solutions.

Machine learning and artificial intelligence can help solve this problem at scale, though. The Wallarm challenge on Kaggle asks participants to help develop more scalable methods to accurately detect attacks and injections through the use of ML and AI.

The competition runs through December 11 and offers $2,000 in prizes.