Register for our next deep dive! Who is Behind Portal Kombat? Exposing the Pravda Disinformation Machine with OSINT on Thursday, June 27, 2024, at 16:00 CET. Grab your spot now! close
17 Nov 2023

Using One Tool to Analyze Internal and OSINT Data Sources

Nitish Chandan

Tasked with sifting through volumes of internal and external data, fraud analysts keep products, organizations, and their users safe by combating scams, abuse, and fraud. The challenge of managing multiple, often disconnected, data sources can complicate their investigations, often leading those teams to wonder if there could be a simpler way—a single platform that would bring all this information together.

However, the choice between adopting an existing tool or creating a custom one is not straightforward and involves weighing immediate functionality against specific needs, cost considerations, and other factors.

This article aims to help you decide whether to build your own tool for investigative data analysis or buy a pre-built one.

We will demonstrate how a unified interface can ease the workload for fraud and threat analysts and explain why a pre-built tool often outperforms the lengthy and resource-intensive process of developing a custom solution in-house. Finally, we will illustrate how Maltego can enhance your manual investigative processes in just a few steps.

Multiple Data Sources Without a Unified Interface Impair Effectiveness 🔗︎

But first, let’s look at the situation where analysts don’t have a unified interface. We will focus on two common scenarios: one where they deal solely with internal data, and another where they handle both internal and external data.

Using Only Internal Data Sources 🔗︎

Ideally, a company would centralize the data for their analysts, simplifying the process of comparison and insight extraction. However, in reality, this ideal is disrupted by rapid organizational growth, frequent changes in toolkits and software, or other factors, leading to data being scattered across different systems.

Consequently, investigators’ operations are hindered by:

  • Fragmented Access: Multiple datasets are not uniform in format or structure, nor do they share a common interface. This fragmentation hinders the ability to quickly and efficiently retrieve the necessary information.
  • Inconsistency: Common issues include varying data types and quality problems, which force analysts to spend considerable time cleaning and normalizing data. This time could be better utilized on more critical tasks.
  • Inefficiency: Often, data retrieval is a manual process that requires significant data processing skills. Sometimes, teams may even need assistance from other departments due to the complexities involved in data storage and access. This inefficiency also impedes collaboration – sharing investigations across team members or working with other teams becomes cumbersome and time-consuming.

Using Both Internal and External Data Sources 🔗︎

Just as reading without glasses could result in blurry and incomplete comprehension, relying solely on internal data can limit an analyst’s ability to fully understand a situation. Without additional context provided by external data sources, crucial details and the broader picture may remain obscured.

To that end, analysts often need to consult external sources such as sanctions data, corporate databases, breach data, and social media data. This enriches their insights into scams, incidents, or broader trends but poses several challenges:

  • Workload: Retrieval from various data sources is often a manual, browser-based process. Although digital risk and brand protection teams typically have standard operating procedures (SOPs) for such investigations, including a list of sources to consult or a process to follow when encountering or investigating a particular data type, these procedures add multiple layers of tasks, thereby prolonging the process and diverting attention from more critical investigative work.

If you wish to save time on manual and repetitive tasks and focus on more meaningful investigative work, we have prepared a guide for you. Download your copy and learn how to automate your investigations with Maltego Machines now.

  • Analyst Exposure: Investigations involving scam websites or social media can risk adversaries gaining information about the teams. Even when practicing operational security by using measures like sock puppet accounts, proxies, and VPNs, these are not foolproof and always carry the risk of exposing sensitive information such as IP addresses, device information, or user agents.
  • Limited Data Access: While analysts utilize OSINT, they also require data from other sources about companies and individuals. Access to these sources, often through browsers or APIs, adds complexity and workload to the integration process, especially when accessing new providers.

Collaboration Challenges 🔗︎

The absence of a unified tool causes problems when handling multiple internal and external sources, but it also leads to inefficiencies in the daily collaboration challenges of investigation teams.

To understand the complexity of their work, let’s step into the shoes of these analysts and view what their work involves:

An everyday task is writing and running complex queries to retrieve data from multiple databases. Once this data is obtained, the next step is to clean it and ensure its quality for reliable analysis and decision-making. This often requires using another tool to manipulate, process, and visualize the data, adding an extra layer to an already intricate process.

To obtain this data in the first place, accessing data sources through APIs or browsers is necessary, which requires adaptability to even more platforms and their unique interfaces.

Finally, investigators are usually responsible for writing detailed reports on their findings, often accompanied by screenshots for evidence. When handing off investigations to other team members, they must write comprehensive mid-investigation reports detailing all steps taken to ensure continuity and clarity.

Maltego allows you to export data from your graph and generate reports automatically, which you can share with your peers and supervisors. Read this article to discover five methods for generating reports in Maltego.

A Unified Tool for Analyzing Diverse Data Sources Enhances Effectiveness 🔗︎

The challenges mentioned highlight the need to adopt a unified tool or interface to streamline the investigative process.

It’s important to keep in mind that this is not just about improving the work of a single analyst, but also about enhancing the success of the investigation and the efficiency of the team as a whole in a number of ways:

  1. Time Efficiency: Utilizing a unified tool simplifies data pulls, making them more straightforward and less time-consuming. This alleviates complex workflows and frees up valuable time for investigators.
  2. Ease of Data Access: Having both internal and external data available through a single interface streamlines the process. This not only makes data processing easier but also reduces the learning curve for investigators, allowing them to adapt more quickly.
  3. Simplified Data Quality Management: With a unified tool, the complexity of ensuring data quality is significantly reduced. Investigators no longer need to clean or normalize data extensively. They can combine internal and external information and search in one place, making their investigations more effective.
  4. Data Protection: If the tool also aggregates data, it can protect investigators’ sensitive information as it acts as a proxy, mitigating the risk of exposure during the investigative process.
  5. Scalability and Efficiency: The ability to collate and analyze data in a single interface facilitates scaling investigations on a large scale without adding to the investigators’ workload. This approach also lends repeatability and consistency to the investigations.
  6. Enhanced Reporting: A unified tool enables one-stop tracking of workflows. Investigations can be easily reproduced, verified, and made compliance-ready, improving the overall quality and reliability of reporting.
  7. Improved Collaboration: The tool allows multiple investigators to work in tandem and track each other’s work efficiently. This feature fosters better collaboration and ensures smoother handoffs between team members.

Building Your Own Tool vs. Buying a Pre-Built Tool 🔗︎

When your team recognizes the pressing need for a single collaborative tool to access all data sources, you start contemplating whether to build an in-house tool or opt for a ready-made solution.

On one hand, developing your own tool should mean that it meets all your specific requirements and fulfills your team’s checklist. On the other hand, ready-made solutions already exist in the market, allowing for immediate implementation and benefits.

  • Building a Custom Tool requires a company-wide effort to restructure and clean existing databases, upskill end-users, and maintain the tool. However, it also risks being a single point of failure if maintenance resources are unavailable, and ensuring compatibility with both internal and external systems is essential. Non-personnel and technical costs further add to the overall expenses.
  • Buying a Pre-Built Tool offers faster deployment and immediate use, with vendors handling technical maintenance and updates, thus reducing the load on internal teams. These tools often come with expert support to aid in implementation and usage. Although the initial cost might be higher than that of custom solutions, long-term costs are often lower. There may be potential limitations in customization, which necessitates looking for a vendor that designs solutions with flexibility and scalability in mind to accommodate organizational growth and evolving needs.

Making this choice requires a careful evaluation of your organization’s resources, needs, and long-term strategy. It’s not a decision to be made overnight but it is essential for the success of your investigations and for effectively addressing security and compliance concerns.

If you’re planning a session with your team to discuss both options, take the ready cheat sheet that we prepared for you to ensure you cover all points.

Accelerating Your Investigations with a Pre-Built Tool 🔗︎

Maltego is an example of a pre-built tool used by analysts coming from Fraud Detection, Digital Security, and Trust and Safety teams. Some of the typical investigations that it supports are:

Its capabilities enable investigators to explore and connect large sets of internal and external data and conduct link analysis using the so-called Transforms which fetch and process data from various sources, converting it into insightful visualizations.

Within a single interface, Maltego allows you to visualize complex data relationships from open sources, internal databases, and even the deep and dark web and enables multiple investigators to work on the same case simultaneously. Furthermore, it includes tools for generating reports, which aids in presenting findings coherently and professionally.

Finally, Maltego can significantly reduce your workload through custom Maltego Machines that let you automate nearly any repetitive steps in investigations like Brand Protection, Person of Interest, and Resale of Counterfeit Goods. For more detailed, ready-to-use steps to automate these types of investigations, check this guidebook.

Download Your Cheat Sheet 🔗︎

If you haven’t done so already, remember to download your comparison cheat sheet for building a custom tool versus buying one for analyzing both internal and external databases for investigations.

Feel free to use it when discussing both options with your team. This cheat sheet should help you cover all the key points.

Download the resource

DE +49
Albania +355
Algeria +213
Andorra +376
Angola +244
Anguilla +1264
Antigua And Barbuda +1268
Argentina +54
Armenia +374
Aruba +297
Australia +61
Austria +43
Azerbaijan +994
Bahamas +1242
Bahrain +973
Bangladesh +880
Barbados +1246
Belarus +375
Belgium +32
Belize +501
Benin +229
Bermuda +1441
Bhutan +975
Bolivia +591
Bosnia and Herzegovina +387
Botswana +267
Brazil +55
Brunei Darussalam +673
Bulgaria +359
Burkina Faso +226
Burundi +257
Cambodia +855
Cameroon +237
Canada +1
Cape Verde +238
Cayman Islands +1345
Central African Republic +236
Chile +56
China +86
Cote d'Ivoire +225
Colombia +57
Comoros +269
Congo +242
Cook Islands +682
Costa Rica +506
Croatia +385
Cuba +53
Cyprus +90392
Czech Republic +42
Denmark +45
Djibouti +253
Dominica +1809
Dominican Republic +1809
Ecuador +593
Egypt +20
El Salvador +503
Equatorial Guinea +240
Eritrea +291
Estonia +372
Ethiopia +251
Falkland Islands (Malvinas) +500
Faroe Islands +298
Fiji +679
Finland +358
France +33
French Guiana +594
French Polynesia +689
Gabon +241
Gambia +220
Georgia +7880
Germany +49
Ghana +233
Gibraltar +350
Greece +30
Greenland +299
Grenada +1473
Guadeloupe +590
Guam +671
Guatemala +502
Guinea +224
Guinea-Bissau +245
Guyana +592
Haiti +509
Honduras +504
Hong Kong +852
Hungary +36
Iceland +354
India +91
Indonesia +62
Iran, Islamic Republic of +98
Iraq +964
Ireland +353
Israel +972
Italy +39
Jamaica +1876
Japan +81
Jordan +962
Kazakhstan +7
Kenya +254
Kiribati +686
Korea, Democratic People's Republic of +850
Korea, Republic of +82
Kuwait +965
Kyrgyzstan +996
Lao People's Democratic Republic +856
Latvia +371
Lebanon +961
Lesotho +266
Liberia +231
Libyan Arab Jamahiriya +218
Liechtenstein +417
Lithuania +370
Luxembourg +352
Macao +853
Macedonia, the former Yugoslav Republic of +389
Madagascar +261
Malawi +265
Malaysia +60
Maldives +960
Mali +223
Malta +356
Marshall Islands +692
Martinique +596
Mauritania +222
Mauritius +230
Mayotte +269
Mexico +52
Micronesia, Federated States of +691
Moldova, Republic of +373
Monaco +377
Mongolia +976
Montserrat +1664
Morocco +212
Mozambique +258
Myanmar +95
Namibia +264
Nauru +674
Nepal +977
Netherlands +31
New Caledonia +687
New Zealand +64
Nicaragua +505
Niger +227
Nigeria +234
Niue +683
Norfolk Island +672
Northern Mariana Islands +670
Norway +47
Oman +968
Pakistan +92
Palau +680
Panama +507
Papua New Guinea +675
Paraguay +595
Peru +51
Philippines +63
Poland +48
Portugal +351
Puerto Rico +1787
Qatar +974
Reunion +262
Romania +40
Russian Federation +7
Rwanda +250
San Marino +378
Sao Tome and Principe +239
Saudi Arabia +966
Senegal +221
Serbia +381
Seychelles +248
Sierra Leone +232
Singapore +65
Slovakia +421
Slovenia +386
Solomon Islands +677
Somalia +252
South Africa +27
Spain +34
Sri Lanka +94
Saint Helena +290
Saint Kitts and Nevis +1869
Saint Lucia +1758
Sudan +249
Suriname +597
Swaziland +268
Sweden +46
Switzerland +41
Syrian Arab Republic +963
Taiwan +886
Tajikistan +7
Thailand +66
Togo +228
Tonga +676
Trinidad and Tobago +1868
Tunisia +216
Turkey +90
Turkmenistan +993
Turks and Caicos Islands +1649
Tuvalu +688
Uganda +256
United Kingdom +44
Ukraine +380
United Arab Emirates +971
Uruguay +598
United States +1
Uzbekistan +7
Vanuatu +678
Holy See (Vatican City State) +379
Venezuela +58
Viet Nam +84
Virgin Islands, British +84
Virgin Islands, U.S. +84
Wallis and Futuna +681
Yemen +967
Zambia +260
Zimbabwe +263

By clicking on "Access", you agree to the processing of the data you entered and you allow us to contact you for the purpose selected in the form. For further information, see our Data Privacy Policy.

Don’t forget to follow us on Twitter, LinkedIn, and Mastodon, and sign up to our email newsletter, so you don’t miss out on updates and news!

Happy investigating!

About the Author 🔗︎

Nitish Chandan 🔗︎

Nitish Chandan is an experienced trust and safety professional with expertise in product risk assessment, crisis response, and investigations. He is currently a Subject Matter Expert at Maltego, where he works with platforms to support their trust and safety investigation needs by developing use cases and investigation pathways. He also regularly conducts training for trust and safety teams to build capacity in risk assessments and fraud investigations. In his last role at Twitter, he was a Senior Product Trust Partner, focused on Twitter’s revenue products and product compliance in the JAPAC region.

By clicking on "Subscribe", you agree to the processing of the data you entered and you allow us to contact you for the purpose selected in the form. For further information, see our Data Privacy Policy.