Tag: open source intelligence

Twitter Analysis as a Tool in Libyan Engagement

The NATO-led Libyan campaign has increased the monitoring of Twitter and other social media in its mission planning, according to today’s Financial Times. Because there are “too few special forces on the ground”, NATO “will take information from every source we can”, according to RAF Wing Commander Mike Bracken, the Libyan operation’s military spokesman. The article even quotes Twitter user @4libya, who on Tuesday tweeted to @NATO what she claimed were coordinates for Gaddafi forces. Without discussing whether this information was used or not, the article goes on to discuss both the advantages and potential pitfalls of using social media in open-source analysis.

Of the many potential pitfalls of canvassing social media, one barely touched on in the article is the simple fact that there is just too much of it. The firehose of information continues to gush, and any attempt to drink it all will quickly overwhelm a system. As such, an analyst needs a tool that highlights the relevant nuggets, pointing at what needs to be read before anything else. This is where Recorded Future comes in.

We chose thirty Libya-based Twitter users and collected the past three months of their tweets. We then spun up a local instance of the Recorded Future platform, which extracts not just entities and events but statements about time. By harvesting this dataset, we were able to see not just who and what the Twitter users were talking about, but when those things were going to happen. This produced some interesting results.

The initial dataset consisted of about 30,000 tweets. Even at only 140 characters each, that’s still a lot to read. Of those tweets, 6,840 mentioned a point in time. The overwhelming majority of those time mentions were reporting on things that happened in the past: last night, last week, last Tuesday, etc. However, 144 of those tweets, or about 1.5%, mentioned a time point in the future: tomorrow, next week, next Tuesday, etc. In effect, they were reporting on something that was going to happen. Suddenly, that giant 30,000-tweet dataset has been immediately triaged to a subset that can be read in minutes.

Recorded Future also allows a deeper dive into the dataset. It’s easy enough to determine, for example, which users are tweeting the most often, and which are tweeting the most often about events to happen in the future. The following graphic, created in Spotfire, shows tweets per day by user, with future-looking tweets in pink:

Some people tweeted more as time went along, some fell off. Some people started making more predictions over time, some made less. Recorded Future lets the analyst quickly visualize both, as well of course allowing them to drill down to the actual underlying data.

For instance, suppose an analyst wants to focus specifically on one user’s forward looking predictions. Easy enough in Recorded Future to pull that data:

By harvesting a single source and using the Recorded Future technology to extract time points, we can easily look at all future-looking statements made by a single user. This can assist in model building, in credibility scoring, and in mission planning.

The analysis of open-source social media can be a great tool to use, and the NATO mission in Libya shows that forces are attempting to use it. The first problem anyone will face is how to deal with the huge amount of data. The Recorded Future platform, available for government customers to be installed behind a private firewall, can take this mass of unstructured text and make sense of it, aligning events and entities across time.

Webinar: Recorded Future + TIBCO Spotfire for Intelligence Analysis

Recorded Future and TIBCO Spotfire will be presenting a joint capabilities demonstration via web conference on Tuesday, March 29. In this session, you’ll learn how analysts can combine Recorded Future’s temporal analytic data with Spotfire’s information visualization technology for government, national security and defense research.

Live Webcast

Date: Tuesday, March 29, 2011
11am Eastern; 8am Pacific
1 Hour
Register Online

OSINT Analysis: Bahrain Unrest Case Study

As of March 9th, oil prices were $104 per barrel and the “market sentiment [was] dominated by concern over unrest in the Middle East”. The financial industry is concerned about events in the Middle East, and market uncertainty has driven up prices up. How can investors and the public navigate this uncertainty? The answer is a combination of international relations logic and open source intelligence.

There is a recurring narrative to the protests in the Middle East, and one of the dominant factors in determining political outcomes is the relationship between the military and the people. On this blog, we’ve analyzed the relationship in Egypt, Libya, and Iran. However, knowledge of the international system alone is not enough to make reasonable predictions. OSINT tools like Recorded Future provide the “meat and potatoes” of the analysis that can support or refute a hypothesis.

In Egypt, intelligence suggested stability because of a fairly close relationship between the people and the military whereas civil war seemed likely in Libya because of clan rivalry. In Iran, Ayatollah Khamenei was likely to brutally oppress the opposition and face little resistance while March 9th, the Washington Post published an article outlining Saudi Arabia’s likely stability. Thus far, these hypotheses have largely held despite a rapidly changing security situation.

OSINT Process

What is the process of open source intelligence gathering? How can one use Recorded Future to make reasonable predictions about political events?

The first step is to pick a topic of interest. In this scenario we will use the Kingdom of Bahrain. The tiny island nation has been in the news as King Hamad bin Isa Al Khalifa clings to power despite sizable democracy protests. The situation in Bahrain is important to the region for geopolitical, religious, and security reasons.

The Saudi government has been watching its neighbor particularly closely as the Shiite majority in Bahrain is demanding reforms similar to calls for protest in Saudi Arabia. Another fear is that hundreds of thousands fleeing Bahrain over the King Fahd Causeway could cause a refugee crisis.

The next step in our analysis is to gather some basic information about the relationship between the military and people in Bahrain. We will use this information to formulate a hypothesis about political outcomes.

News coverage suggests that the military is loyal to King Hamad bin Isa Al Khalifa and fired on protesters. Important to consider as recent events show continued stability for military-backed governments although there are complicating religious factors that make Bahrain harder to predict.

What news items support or refute the hypothesis of a stable Bahrain? Let’s gather some intelligence.

Timeline of Bahrain Protests

Above is a timeline of the protests in Bahrain, from when they first began to gain momentum in mid-February until now. Some items of note include the following:

  • The Shiite opposition has effectively left the government. This establishes that opposition groups exist in Bahrain and were members of parliament. Even more substantial, the majority of the populace in Bahrain is Shiite although the ruling family is Sunni. This religious difference could stir sectarian conflict in Bahrain.
  • The momentum of the protests tanked after February 17th, around the time when several protesters were killed. It appears that the people did not continue mass protests in the face of a government crackdown. Protests may still be ongoing, but they are not nearly at the levels they were in mid-February.

The protests in Bahrain appeared to be vicious in mid-February. So where did they go? What did King Al Khalifa say during this period?

Overview of Al Khalifa's Statements

  • It seems that there is a stalemate in Bahrain. The opposition groups have pulled out, but the King stated that he would open a dialogue. The Shiite opposition groups have refused to talk until the government resigns.

Timeline of Al Khalifa's Statements and Concessions


One article that Recorded Future’s timeline tool uncovered was a New York Times article on the role of the military in Arab governments. It talks in detail about the relationship between the military and the people in Egypt and Bahrain. The author notes, “as in nearly all police states, the key to change lies with the military.” In contrast to Egypt, “the military [in Bahrain] seems to have concluded that adapting to change would do them no good — that the protesters were far too great a threat to their very command of society,”

Final Analysis

The primary factor supporting the stability hypothesis is that the military in Bahrain firmly supports King Al Khalifa. The bedrock of stability in a police state is the military, and with control of this apparatus King Al Khalifa has the advantage. Another reason to support the stability hypothesis is that the momentum of the protests has largely faded. The opposition does not appear to be as strong as it was in mid-February.

However, the fact that the majority Shiite opposition has withdrawn from the government and is refusing to talk is troubling. The fact that the Al Khalifa government is minority Sunni and the people are majority Shiite could raise the specter of sectarian conflict. An influx of arms from another actor in the region could change the balance of power rather quickly.

Interested in conducting similar analysis or leveraging our news analytic data for OSINT? Contact our Federal Team today!

Hizb ut-Tahrir: Peaceful or Plotting?

In a recent post, we discussed Yemeni opposition parties and their ties to extremist groups. A few days ago, the most influential leader of the Islah party Abdul Majid al-Zindani stated that “an Islamic state is coming” in Yemen. Abdul Majid al-Zindani is a former mentor of Osama Bin Laden and the leader of the Islah party representing Islamists, tribalists, and Salafists in Yemen. He is also one of the most powerful clerics in Yemen and his words carry considerable weight.

There is another growing Islamist force on the rise in Yemen and elsewhere. That force is Hizb-ut-Tahrir, a pan-Islamic organization that opposes democracy and seeks the “formation of a transnational Islamic caliphate”. It is banned in many countries and has been accused of supporting terrorist organizations. However, Hizb-ut-Tahrir claims that it is a peaceful organization of one million members and does not support violence.

Using open source intelligence tools we can discover more about the political affiliations and statements of Hizb-ut-Tahrir. Is the organization really what it claims to be?

Aside: For general background information on Hizb-ut-Tahrir, consider reading GlobalSecurity.org’s article on the party. It provides a foundational introduction to the organization’s political structure and motivations.

So, who leads Hizb-ut-Tahrir? The leader of an organization is almost always at the top of the policy pyramid and plays a critical role in shaping its ideology. Recorded Future’s network tool can provide information about the political statements of Hizb’s leaders.

Hizb-ut-Tahrir Link to Burhan Hanif

One of the leaders of Hizb-ut-Tahrir is Burhan Hanif, who has openly stated that, “Muslims should shun democracy.” Most of what Hizb-ut-Tahrir has said in the press also reflects this viewpoint. Over the past few years we find a steady flow of information supporting Hizb-ut-Tahrir’s anti-democratic sentiment.

Hizb-ut-Tahrir Timeline of Quotations Past Year

Points of interest include:

  • In July, the Marriott in Chicago canceled a conference planned by Hizb-ut-Tahrir when it discovered that, “speakers at previous conferences have denounced democracy”. A Muslim school in Bridgeview, Illinois, also backed out of hosting the group. The article also indicates that Hizb-ut-Tahrir has a relatively pale following in the United States compared to other Muslim groups.
Statements and Political Events Related to Hizb-ut-Tahrir

Hizb-ut-Tahrir Media Overview

  • An editorial by the Telegraph of London notes that, “Hizb-ut-Tahrir is an organisation that is anti-semitic and anti-democratic. Its members in Denmark have called for the killing of Jews and one of its activists here has called for the death of the criminal capitalist nation of America and other infidel (kuffar) states by the army of jihad”. It also states that Hizb-ut-Tahrir publications have described “Western education as a threat to our [Muslim] beliefs and values”.
  • The group was nearly banned in Australia because of its political platform and possible links to terrorist groups. Although these links have not been completely confirmed, Clive Williams, head of terrorism studies at the Australian National University indicates that, “There are many instances, though, of those whose views were forged in Hizb ut-Tahrir subsequently taking part in terrorism.” It has been reported that the group also supports the insurgencies in Iraq and Afghanistan and calls for the military elimination of Israel.  The group has also been banned in Russia and listed as a terrorist organization.

Does Hizb-ut-Tahrir have any links to potentially violent groups?

Links to Violent Groups

Hizb-ut-Tahrir is suspected of having links to al-Muhajiroun and the Islamic Movement of Uzbekistan. Hizb is currently banned in several central Asian countries, perhaps based partially on these links. Al-Muhajiroun is a banned Islamist organization in Britain and is proscribed under the UK Terrorism Act of 2000. The IMU was a terrorist organization that sought to overthrow the government of Uzbekistan and implement Sharia law. It is thought that the organization no longer exists.

Where does Hizb-ut-Tahrir fit into the current crisis in the Middle East?

The organization recently has shown its ability to flex its political muscle at the university level. In Bangladesh, “Hizb-ut Tahrir – which has well known global links – has been active lately in universities and the government may well have been worried about its increasing influence among students”. This ability to organize may have a future impact on events in the region.

The current protest movements have largely been influenced by students, and Hizb-ut-Tahrir has shown its ability to appeal to at least a portion of the student populace. Although the group may not have overt political goals (such as running for election) it still may be able to influence political events behind the scenes.


OSINT evidence suggests that Hizb-ut-Tahrir is staunchly anti-democratic and may be a security threat to regional governments. It is currently banned in a number of countries for this reason. Hizb-ut-Tahrir may still attempt to exploit the current political situation in the Middle East, even though most protests so far support democratic reform. Its anti-democratic message may appeal to a subset of the population, particularly in countries such as Yemen that lack secular opposition parties.

Further, the group has shown its ability to organize at the university level and appeal to a younger audience. In Yemen and other failing Middle Eastern governments, it may take advantage of a newfound opportunity to spread its ideology.

Open Source Intelligence: Back to the Future(!)

Time Line Intelligence Analysis Tools One of the beauties of building the world’s largest temporal event database is that we can actually know what the view of the future was like at a certain point in time! When Recorded Future collects data, we automatically record when published, analyzed, and of course, what time point in the past/present/future it’s associated with.
You may wonder how this can be helpful? Here are two good examples of how it makes for efficient open source intelligence analysis.

On the 12th of July, Al-Shabaab, the Somalia-based terrorist organization, blew up multiple bombs in Kampala, Uganda.  Assume we want to look at the world from the view of July 11th, 2010, the day before the bombing.  A new Recorded Future feature (to be released shortly!) allows a user now to not only specify Event time but also Publishing time. Below we can look for quotations by the head of Al-Shabaab – Mukhtar Abdurahman Abu Zubeyr – on Uganda – before July 11th, and find an article referring to a week old audio tape where he pretty bluntly threatens Uganda and Burundi.

 Al-Shabaab – Mukhtar Abdurahman Abu Zubeyr Intelligence Analysis

Zubeyr Statements

By July 13th, this particular news item published July 5th had been replicated thousands of times around the world.

We can go a bit broader.  By looking for anyone speaking about Uganda, but with publication time constrained from say June 1st to July 11th, we find a broader but still manageable universe  of materials. In the timeline view we can review this, and see which of these fall in the past/present/future.

Time line Intelligence Analysis

Timeline View

Finally, we can look for indications of a person’s prediction ability.  For example, we’ve earlier reported on John Gruber’s good insights on the iPhone 3Gs name.  In this case, however, by looking for who spoke about the iPad ahead of Jan 26th, 2010, we find that John Gruber explicitly spoke against the iPad name, whereas Arnold Kim from MacRumours says that an Apple dummy company had been filing trademark applications for “iPad” around the world.
Open Source Intelligence Time Line

Using Publish Time

How can this be helpful for your work? Enjoy!

You can learn more about our news analytics API and Recorded Future applications for intelligence analysis on our web site.

Open Source Timelining: David Coleman Headley

There has a been lot of discussion recently about David Coleman Headley (formerly known as Daood Sayed Gilani), between his involvement in multiple very high profile terrorist attacks as well as interactions between US and Indian intelligence agencies (such as NIA).

To get a sense of Mr Headley we took a quick look at the Recorded Future index. We can observe that our content starts early in the 1960s, with Mr Headly being born, but content grows dramatically later. We can also see in the momentum sparkline just under David Coleman Headley’s name that recent interest in him has grown strongly.

Zooming in a bit we find that this guy is no “Mr nice guy” – highlighting here charges of possession of heroin in 1997.

We might specifically be interested in his travel patterns – and can easily examine those:

And if we want examine the actual relationships between people, places, etc. involved in his travels we can easily change to the network view:

Likewise, we can examine people which David Coleman has communicated with through the Person Communication event – yielding this network – showing all kinds of “interesting” people in David Coleman’s network.

In conclusion – open source intelligence analysis with Recorded Future can very quickly provide a sense of a person, even with a very long period of complex events and relationships involved.

Copyright © 1996-2010 Analysis Intelligence. All rights reserved.
iDream theme by Templates Next | Powered by WordPress