The Newsbeuter RSS Feedreader

"Newsbeuter" is a pun on the German word "Wildbeuter", which means "hunter-gatherer". During the stone age, people hunted and gathered their food, and these days, they hunt and gather news and information. Credits for this idea goes to Clifford Wolf, who submitted it to a little competiton that was started when I got aware that the original name would violate French and European registered trademarks.

Newsbeuter is available as source package. Simply go to http://synflood.at/newsbeuter.html and download the latest source package, which is usually in the .tar.gz file format. Alternatively, you can check out the latest development source tree from the newsbeuter SVN repository by running the following command on the commandline:

svn co http://bereshit.synflood.at/svn/noos/trunk

Newsbeuter depends on a number of libraries to function correctly. This table lists these dependencies. Please be aware that the list libraries may themselves depend on other libraries. These dependencies are not listed here. Please also be aware that you need a recent C++ compiler. Currently, newsbeuter has only been tested with GCC.

If you intend to modify the filter language parser, you will also need Coco/R for C++, which you can download from http://www.ssw.uni-linz.ac.at/coco/. The Coco/R binary must be installed as "coco-cpp" in your PATH. Debian users only need to install the package "coco-cpp".

After you've downloaded and installed the dependencies mentioned above, you can start compiling and installing newsbeuter. To compile newsbeuter, simply run "make" in the source tree. After a short time, this should complete successfully, and you can go on with installation by running "make install". By default, this will install the "newsbeuter" binary to the /usr/local/bin directory. You can provide an alternative installation path using the prefix parameter, e.g. running "make install prefix=/opt/newsbeuter" will install the binary to the directory /opt/newsbeuter/bin.

# a comment
max-items        100 # such comments are possible, too
browser          links
show-read-feeds  no

unbind-key       R
bind-key         ^R    reload-all

It is possible to configure custom color settings in newsbeuter. The basic configuration syntax is:

color <element> <foreground color> <background color> [<attribute> ...]

This means that if you configure colors for a certain element, you need to provide a foreground color and a background color as a minimum. The following colors are supported:

The "default" color means that the terminal's default color will be used.

Optionally, you can also add one or more attributes. The following attributes are supported:

Currently, the following elements are supported:

The default color configuration of newsbeuter looks like this:

background   white   black
listnormal   white   black
listfocus    yellow  blue   bold
info         yellow  blue   bold
article      white   black

Newsbeuter comes with the possibility to categorize or "tag", as well call it, RSS feeds. Every RSS feed can be assigned 0 or more tags. Within newsbeuter, you can then select to only show RSS feeds that match a certain tag. That makes it easy to categorize your feeds in a flexible and powerful way.

Usually, the _{/.newsbeuter/urls file contains one RSS feed URL per line. To assign a tag to an RSS feed, simply attach it as a single word, separated by blanks such as space or tab. If the tag needs to contain spaces, you must use quotes (") around the tag (see example below). An example} /.newsbeuter/urls file may look like this:

http://blog.fefe.de/rss.xml?html interesting conspiracy news "cool stuff"
http://rss.orf.at/news.xml news orf
http://www.heise.de/newsticker/heise.rdf news interesting

When you now start newsbeuter with this configuration, you can press "t" to select a tag. When you select the tag "news", you will see all three RSS feeds. Pressing "t" again and e.g. selecting the "conspiracy" tag, you will only see the http://blog.fefe.de/rss.xml?html RSS feed. Pressing "^T" clears the current tag, and again shows all RSS feeds, regardless of their assigned tags.

From version 0.4 on, newsbeuter contains support for Snownews extensions. The RSS feed readers Snownews and Liferea share a common way of extending the readers with custom scripts. Two mechanisms, namely "execurl" and "filter" type scripts, are available and supported by newsbeuter.

An "execurl" script can be any program that gets executed and whose output is interpreted as RSS feed, while "filter" scripts are fed with the content of a configured URL and whose output is interpreted as RSS feed.

The configuration is simple and straight-forward. Just add to your ~/.newsbeuter/urls file configuration lines like the following ones:

exec:~/bin/execurl-script
filter:~/bin/filter-script:http://some.test/url

The first line shows how to add an execurl script to your configuration: start the line with "exec:" and then immediately append the path of the script that shall be executed. If this script requires additional parameters, simply use quotes:

"exec:~/bin/execurl-script param1 param2"

The second line shows how to add a filter script to your configuration: start the line with "filter:", then immediately append the path of the script, then append a colon (":"), and then append the URL of the file that shall be fed to the script. Again, if the script requires any parameters, simply quote:

"filter:~/bin/filter-script param1 param2:http://url/foobar"

In both cases, the tagging feature as described above is still available:

exec:~/bin/execurl-script tag1 tag2 "quoted tag"
filter:~/bin/filter-script:http://some.test/url tag3 tag4 tag5

A collection of such extension scripts can be found on this website: http://kiza.kcore.de/software/snownews/snowscripts/extensions

If you want to write your own extensions, refer to this website for further instructions: http://kiza.kcore.de/software/snownews/snowscripts/writing

Like other text-oriented software, newsbeuter contains an internal commandline to modify configuration variables ad hoc and to run own commands. It provides a flexible access to the functionality of newsbeuter which is especially useful for advanced users.

To start the commandline, type ":". You will see a ":" prompt at the bottom of the screen, similar to tools like vi(m) or mutt. You can now enter commands. Pressing the return key executes the command (possibly giving feedback to the user) and closes the commandline. You can cancel entering commands by pressing the ESC key. Currently, the following commands are available:

Newsbeuter provides a powerful filter language that enables the user to filter the content of many dialogs, such as the feed list or the article list. The basic concept is that every feed and every article has a number of attributes which can then be compared with user-supplied values, and these comparisons and be logically AND'ed, OR'ed and grouped.

Examples for simple filter expressions are:

unread_count > 0
rssurl =~ "^https:"

Logically connecting and grouping such expressions looks like in the following examples:

( unread_count > 0 and unread_count < 10 ) or total_count > 100
( author =~ "Frank" or author =~ "John" ) and ( title =~ "Linux" or title =~ "FreeBSD" )

The possibilities for combining such queries is endless, sky (actually: the available memory) is the limit.

To filter your feeds, press "F" in the feed list, enter your filter expression, and press enter. To clear the filter, press Ctrl-F. To filter the articles in the article list, press "F", enter your expression, and press enter. Clearing the filter works the same as before. Be aware that only certain attributes work in both dialogs. The table below lists all available attributes and their context, i.e. an attribute that belongs to a feed can only be matched in the feed list, while an attribute that belongs to an article can only be matched in the article list.

Note that it's also possible to filter for feed attributes when you query for article attributes. This is because every article is internally linked to the feed from which it was downloaded.

Sometimes, a user is confronted with certain content he doesn't want to read, e.g. on topics the user is not interested in or articles from certain people he doesn't want to read. In Usenet, such functionality within software is traditionally called a "killfile", i.e. based on the content of this "killfile", articles that match certain conditions do not get displayed and are not presented to the user at all.

In newsbeuter, such a "killfile" can be implemented on a per-article basis via the configuration file. The most important configuration command for this is "ignore-article":

ignore-article "*" "title =~ \"Gentoo\""
ignore-article "http://synflood.at/blog/index.php?/feeds/index.rss2" "title =~ \"newsbeuter\""

The basic format is that the user specifies an RSS feed for which the ignore shall be applied ("*" matches all RSS feeds), and then a filter expression (see previous section). If newsbeuter hits an article in the specified RSS feed that matches the specified filter expression, then this article is ignored and never presented to the user. The configuration itself can contain as many ignore-article commands as desired.

Query feeds are a mechanism of newsbeuter to define custom "meta feeds" by using newsbeuter's built-in filter language. A query feed is a feed that is aggregated from all currently downloaded articles of all feeds. To narrow down the set of articles, the user has to specify a filter. Only articles that match this filter are added to the query feed. A query feed is updated whenever it is entered in the feed list. When you change the unread flag of an article, this is reflected in the feed where the article was originally fetched.

To define a query feed, the user has to add a line to the file ~/.newsbeuter/urls in the following format:

query:<name of feed>:<filter expression> [<tag> ...]

The "query:" in the beginning tells newsbeuter that it's a query feed, "<name of feed>" specifies the name under which the query feed shall be displayed in the feed list, and "<filter expression>" is the filter expression that shall be used. Like every other feed, a query feed can be tagged to organize it like a regular feed.

A good example for the user of this feature is a query feed that contains all unread articles:

"query:Unread Articles:unread = \"yes\""

Note the quotes that are necessary around the complete query "URL" and the backslashes that are necessary the escape the quotes in the filter expression.

If you want to combine several feeds to one single feed, a good solution is to tag the feeds that you want to combine with one certain tag, and then create a query feed that only displays articles from feeds with that certain tag:

http://domain1.tld/feed.xml fun news tag1
http://domain2.tld/?feed.rss private jokes tag1
http://domain3.tld/feeds.rss news
"query:tag1 Articles:tags # \"tag1\""

In this example, the feeds http://domain1.tld/feed.xml and http://domain2.tld/?feed.rss are aggregated into the query feed named "tag1 Articles", but the feed http://domain3.tld/feeds.rss is not.

Basically, the possibility of what can be realized with query feeds is only limited by what can be queried from articles and feeds with the filter language and by your creativity.

A podcast is a media file distributed over the internet using syndication feeds such as RSS, for later playback on portable players or computers. Newsbeuter contains support for downloading and saving podcasts. This support differs a bit from other podcast aggregators or "podcatchers" in how it is done.

Podcast content is transported in RSS feeds via special tags called "enclosures". Newsbeuter recognizes these enclosures and stores the relevant information for every podcast item it finds in an RSS feed. What the user then can do is to add the podcast download URL to a download queue. Alternatively, newsbeuter can be configured to automatically do that. This queue is stored in the file $HOME/.newsbeuter/queue.

The user can then use the download manager "podbeuter" to download these files to a directory on the local filesystem. Podbeuter comes with the newsbeuter package, and features a look and feel very close to the one of newsbeuter. It also shares the same configuration file.

A usual "use case" is to configure newsbeuter to automatically enqueue newly found podcast download URLs. Then, the user reloads the podcast RSS feeds in newsbeuter, and after that, he/she uses podbeuter to view the current queue, and either selectively download certain files or automatically download them all together by pressing "a" within podbeuter.