Help Bellingcat create tools for open source investigators!
Open source surveys rely on the skills and efforts of human researchers. It will never change. However, there are tools that can help make the research process a lot easier.
At Bellingcat, like many other research organizations and open source enthusiasts, we rely primarily on tools that are available free of charge. On the one hand, it’s because we don’t have the budget to pay for access to many paid tools – we’re a nonprofit, after all. But it’s also because we believe in the idea of open source software.
We want digital researchers from all walks of life to have the methods and tools available to them to be able to conduct open source surveys. That is why we publish free research guides and resource lists on our website. This is also why we have launched a Bellingcat Github, where many tools developed and used by Bellingcat’s Tech forensics team will be available as open source projects.
In addition to releasing these tools, we are also open to contributions and collaborations from those with experience with open source software as well as open source investigative enthusiasts.
How you can help
- Discover the tools on our Bellingcat Github and help us improve them.
- Read our list of open questions and start working on some of the more complex technical challenges that are relevant to open source investigators.
- If, for example, you run a computer science department at a university or work in a technology-driven research institute and have a concrete idea for a tool development collaboration (the software should be open source), don’t do not hesitate to contact us here.
- The same goes for any open source investigator who needs a specific tool for their job. You can submit your ideas here and if we think the development of the tool is feasible, we will add it to our list of technology challenges. You can also submit a “pull request” directly. Maybe a helpful software developer will take care of that.
Use, test and improve our Github tools
The list of tools on our Github will grow over time. Currently, our small collection includes, among other things, a tool for finding Instagram “locations”, an automatic video archiver for Google Sheets, and a Google Earth Engine application for finding small cloudless sub-regions in cloudy satellite imagery.
We hope that with your help we can improve these tools and add more over time. Many of these projects already have suggestions for improvement on their Github Issues pages, including highlighted issues that would be great for community contributions.
Learn more about the tools in our Github to which you can already contribute:
Instagram is somewhat unique among modern social media websites in that public sharing and geolocation are still social norms. However, photos cannot be searched by specific contact details – rather, they are associated with an Instagram “location”. Instagram does not provide a way to search or discover these places by latitude and longitude coordinates, only by city and place name. For example, if it’s easy to find Instagram posts from Amsterdam Central Station, there’s no easy way to find photos taken at other places nearby, unless you know exactly what you’re looking for (and the corresponding Instagram place name).
However, the Instagram mobile app accesses a private API, or application programming interface, which finds places near a particular Instagram location and is used when selecting a location for a new photo. Using this API, Bellingcat has developed a tool for finding Instagram places (as explained and demonstrated in the thread below).
Do you want to know where people tag posts on Instagram in a specific city or town? Try our Python script which finds and maps these locations. (1/7) pic.twitter.com/Abc2lGBX54
– Check it out! cation Quiz Bot (@quiztime) December 16, 2020
Several tools already exist to download images from a particular Instagram location or user. A useful improvement to instagram location search would be to integrate these tools to allow a searcher to retrieve all images around a particular latitude and longitude in one step.
Telegram Phone Number Checker is useful if a researcher wants to know if a specific phone number, or list of numbers, has been used to create accounts on Telegram.
To verify a phone number, the tool uses the Telegram API and requires a ‘‘API_ID’ and ‘API_HASH’. If a phone number is associated with a Telegram username, the tool provides the logged in username. Since usernames are not required on Telegram, it can also just detect if a Telegram account has been created with a specific phone number.
It is a tool which combines youtube-dl with Google Sheets to create a collaborative interface for archiving videos from social networks. It is a Python script which, when executed regularly (as with cron):
- Download recognized videos from URLs in a Google sheet.
- Download these videos to a cloud storage location compatible with Amazon Web Service’s S3 protocol, such as S3 itself or Digital Ocean Spaces.
- Update the sheet with metadata that might be useful for organization and retrieval.
The tool was developed by Bellingcat to handle social media images of the 2021 U.S. Presidential Inauguration, but we believe it could be useful in a variety of other situations.
While most of the metadata is provided directly by youtube-dl, the script can also transform the video format and extract the metadata using ffmpeg, an open source tool for video processing. This is used to generate a thumbnail, which can be inserted directly into the Google Sheet as an image. It can also generate periodic thumbnails for a video “contact sheet” which makes it easier to see video content at a glance.
There are several possibilities to extend the functionality of the auto-archiver and to make it easier for new or non-technical users. Additional guides and examples in the Github documentation would be a great addition and make the app easier to use. Alternatively, there may be other app formats, such as a Google Sheets app, that would make some parts of the setup easier. However, since it requires a remote computer to run youtube-dl and a location to download archived videos, it would likely still require server setup.
This tool is extremely simple helper to extract exact upload date from Tiktok video. the source code shows how to query a website that doesn’t provide CORS headers and how a simple client-side tool can be hosted on Github pages.
This is a Google Earth Engine application that tries to find Sentinel-2 cloudless satellite images in a particular area. Sentinel-2 is an Earth observation satellite operated by the European Space Agency that captures medium-resolution (10 meters per pixel) visible and infrared images of most locations on the Earth’s surface approximately twice per week. This imagery is extremely useful to open source investigators, and in recent articles Bellingcat has been used to understand environmental pollution in Iraq and analyze allegations of drone war in Tigray, Ethiopia.
This tool is especially useful in equatorial regions where Sentinel-2 routinely details significant cloud cover when measured over entire images, as provided by default in Google Earth Engine and Sentinel Hub. However, some of these images may have clear skies over particular small areas of interest. This application helps to find these images.
In addition, it filters image collections to contain only one image per month, which simplifies the process of sorting over long periods that may contain many images. It also offers contrast adjustment and false color options in the near infrared which are useful for visualizing vegetation.
It can be accessed directly as Google Earth Engine application.
Contribute to our more complex challenges
We also publish some “open questions”For open source surveys. These are somewhat difficult long-term technical challenges that could be of use to open source investigators.
Although possible solutions to these challenges have been explored within the academic community, there are (to our knowledge) no accessible tool for open source researchers who implement these methods or explore new approaches. These challenges are complex and open-ended, and they might not be resolved in a way that is practical for open source investigators. But we are interested in trying.
Our list of current challenges is posted on GitHub, allowing us to be transparent and collaborative about how we solve them. If you are interested in contributing, we welcome your additions to our repository, which you can submit through a GitHub pull request or by sending us a message via our contact form.
Contributions that would be most welcome include additional ideas for technical challenges, literature exploration notes on related work, and prototypes of potential approaches to the problem. If there is sufficient interest in a particular challenge, it could be drifted into its own project, potentially with the support of Bellingcat’s technical investigative team.
Thank you in advance for your contribution ! Such work will help make life easier for the global community of open source investigators.