Hourly World News Updates

Story Selection

The first step is to aggregate all the world news stories posted by the following newspapers via their RSS feeds;

• New York Times
• Washington Post
• CNN
• Fox News
• BBC
• USA Today
• NBC News
• ABC News
• CBS News
• LA Times
• Bloomberg
• Wall Street Journal
• Reuters
• Associated Press
• Los Angeles Times
• The Guardian
• Bloomberg News
• Huffington Post
• NPR News
• Al Jazeera
• Deutsche Welle
• CBC News
• South China Morning Post
• The Japan Times

These sources represent the most popular news sources in the world. It's true that many of them are deeply biased and problematic. Compensating for that is the whole point of building this tool.

The first step is to remove any unimportant words like prepositions, articles, etc.

Next, we count the frequency of all the words in all the headlines.

Then each word gets a score based on how often it's used. I am continuously refining the list of unimportant words which is visible in the code of this page.

Next, we loop through all the headlines and sum up the scores of all the words. This gives us a score for the headline. They are all sorted by this score, and the top ten are what you see here.

The Future

I have already been experimenting with incorporating in-browser large language models to actively synthesize and distill the content of the headlines. This has other benefits such as intelligently combining related topics.

I'm also working on a branched visualization system which starts with the most commonly occuring word and then branches into the different words that trend with it. This may allow us to see a sort of heatmap of the topics and how they are related.

Stay tuned!