Besides hate speech through text, it is also important to consider hate speech through images – memes. Often the text from the image might not constitute  hate speech on its own , which can also be applied the other way around. The goal is to develop a solution which will be able to detect this type of hate speech.

Before you start developing your idea, you should check developer guidelines and existing tools and resources.


The goal of this hackathon is to have a functional prototype of the solution which will be ready or nearly ready for the testing phase. 

The solution can be add-on or plug-in for public communicators to easily moderate potentially harmful content on social media posts using existing hate speech detection datasets and open software to create an automated hate speech image detection software able to timely warn public communicators on potentially harmful comments online. 


The solution should avoid banning or deleting potentially harmful content. It should also avoid building software only usable in one language with a high barrier to language adaptation.




The following principles stem from issues encountered during previous attempts to analyze and detect hate speech online. Successful solutions should focus on mitigating these issues.


  • Memes are often used as a form of hate speech and are difficult to detect. Remember that a text might not look harmful (I love the way you smell. Look how many people love you). Image might not look harmful (Skunk in nature. Image of wasteland). However, combining harmful images and harmful text can create a hateful  meme.[Feel free to develop an algorithm that replaces the image and changes the sentiment to  a positive one.]

  •  Political memes that are used in a context of a current (past) geo-political situation are very popular. A knowledge of the context is required to define it as a form of hate speech.

  • Emoticons can also be misused to express hate speech because they are excluded from natural language processing methods. Their meaning also requires context to be properly interpreted and classified.



Feel free to choose from any of the examples of widely-used development or machine learning tools and lexicons. The list is not exhaustive and serves as an inspiration. The list was developed based on the Overview of Online Hate Speech Detection Solutions, a document created by Open Code for Hate-free communication. The original resource can be found here.


Development tools, libraries and services

  1. TensorFlow - a library for constructing and training a neural network to automatically identify objects and classify images.

  2. PyTorch - a python package for tensor computation

  3. tracking.js - is a library that brings different computer vision algorithms and techniques into the browser environment.

  4. VisionAI - Google cloud service that lets you derive insights from your images in the cloud or at the edge with AutoML Vision. Alternatively, you can use pre-trained Vision API models to detect emotion, understand text, and more.

  5. OpenCV - highly optimized image recognition library with focus on real-time applications. It supports C++, Python or Java interface.

  6. EmguCV - .NET wrapper for OpenCV image-processing library

  7. VXL - Collection of computer vision libraries

  8. MMF - a modular framework for vision & language multimodal research from Facebook AI Research (FAIR). Build on top of PyTorch

  9. Google Teachable - image classification tool

  10. Emoji library for Python

Data sets and tools

  1. Memotion Datasets - dataset for sentiment classification of memes

  2. Hateful Memes - another dataset of hateful memes

  3. Exploring Hate Speech Detection in Multimodal Publications

  4. Hateful Memes Challenger Facebook AI