The Node

Posts Tagged ‘rss’

With the sad (and annoying) news that Google Reader is to be shut down on July 1st 2013, I had the dismal job of finding a replacement. Then I thought, why not just host my own RSS aggregator service? At least that way, I’m not at the whims of some corporation shutting down the services I use. So here, I’m going to show you how to set up tt-rss using Fedora Linux, Apache MySQL and PHP. This will allow you to import your Google Reader data into tt-rss’s MySQL database and display as a web application on your web server. There’s even an Android app to go with your installation to replace the Google Reader app. Hooray!

Okay, so I’m going to be using Apache, MySQL and PHP under Fedora Linux for this. If you already have Apache, MySQL and PHP installed, skip to the next section. If not, keep reading.

I’ll assume you’re on your Fedora Linux box as the root user for this. To install a basic Apache web server under Fedora with PHP and the libraries tt-rss will need, run the following: -

yum install apache mysql mysql-client php php-xml

Once all that is installed, start up your Apache installation with: -

service httpd start

In order to see web pages, you’ll also need to make sure your firewall is open on port 80. Fedora uses iptables as it’s firewall so let’s open a port for the web: -


iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT


If you’re planning on using an SSL certificate for HTTPS secure connections, you’ll need both commands. If not, you’ll just need the first one that opens port 80. For this (or any) web server to be useful you’ll also need to open the port(s) on your router’s firewall too. Check that your server is listening on the right port with: -

netstat -l | grep http

…which should give you back one or both ports listening as below: -

tcp6       0      0 [::]:http               [::]:*                  LISTEN
tcp6       0      0 [::]:https              [::]:*                  LISTEN

Check that you can reach your server’s ip address (ifconfig will tell you this) by opening a browser and pointing it to “http//your_ip_address”. If all is well, you now have Apache installed and running. To make sure that PHP is installed and working with Apache, create a test PHP page with the following: -

vi /var/www/html/test.php

and put in the following: -


< ?php
phpinfo();
?>


Browse to “http://your_ip_address/test.php” and check that you get the PHP diagnostics page up. If so, you’re ready to install tt-rss. But first, you need to export your Google Reader subscriptions while you still can! If you just want to set up tt-rss without importing from Google Reader, skip this step.

Log in to your Google Reader account here and go to Settings > Reader-Settings -> Import/Export. Click on “Download your data through Takeout” and then click on “Create Archive” which will allow you to download the zip file. Unzip it to a directory on your local hard drive.

Next you need to download tt-rss from the official website. As of the time of writing the current version is 1.7.2. I usually go to where I want it installed first, so: -

cd /var/www/html

..as this is the default location for your web pages, and run: -

wget https://github.com/gothfox/Tiny-Tiny-RSS/archive/1.7.2.tar.gz

If you don’t have wget installed under Fedora, install it with: -

yum install wget

Unpack the archive with: -


gunzip 1.7.2.tar.gz
tar -xf 1.7.2.tar


You should now have a subdirectory called tt-rss or similar. Change to that directory now.

Previously, you installed MySQL and now we need to set up a database under MySQL for tt-rss to store it’s information. If you already have MySQL installed, skip this step.

Start MySQL with: -

service mysql start

If you’ve not used MySQL before, it’ll prompt you to change the root user’s password. Do that now with: -

mysqladmin -u root password NEWPASSWORD

…where “NEWPASSWORD” is the password you want to use. Login to MySQL with: -

mysql -uroot -p

and enter your password. Next we need to set up a user for tt-rss for security’s sake. Run the following SQL: -

GRANT ALL PRIVILEGES ON ttrss.* TO 'rss'@'127.0.0.1' IDENTIFIED BY 'somepassword';

This will add a new MySQL user called “rss” which has complete access to all tables in the “ttrss” database. Their password is “somepassword”. Next, flush the tables with: -

FLUSH ALL PRIVILEGES

Next, we need to create the ttrss database. Do this with: -

CREATE DATABASE ttrss;

and then “exit” out of MySQL.

Try logging into your MySQL account with: -

mysql -urss -psomepassword

…noting the lack of spaces between the -u and -p for username and password respectively.If you can login and see your ttrss database by running: -

SHOW DATABASES;

…you are ready to go. Exit out of MySQL again and change to the schema directory under Apache where you installed tt-rss, usually something like: -

cd /var/www/html/tt-rss/schema

Next we need to import the database tables for MySQL for the tt-rss web application using the provided MySQL scheme SQL file.

mysql -urss -psomepassword ttrss < ttrss_schema_mysql.sql

This should import the tables to your previously created database using the MySQL user "rss" we just set up.

Once this is done, back up a directory level with: -

cd ..

Next we need to copy the default configuration file to something tt-rss can use, so run the following: -

cp config.php-dist config.php

Next, edit the file with vi and change the following directives to your settings: -

define('DB_TYPE', "mysql");
define('DB_HOST', "127.0.0.1");
define('DB_USER', "rss");
define('DB_NAME', "ttrss");
define('DB_PASS', "somepassword");
define('SELF_URL_PATH', "http://your_ip_address/tt-rss/");

Save this file and then try to browse to the URL defined in SELF_URL_PATH. You should be prompted to either unlock the permissions on certain directories or to login. The default username and password is “admin” with “password”. Remember to change them!

Before logging in, we need to choose a method of updating our feeds that we’ll import from the Google Takeout zip file we downloaded. There are two basic ways, via a daemon or via a cron job. The daemon way is recommended and can be done with: -

nohup php /var/www/html/tt-rss/update.php -daemon > /var/log/tt-rss.log &

Here, I’m running the process as a background process as the daemon doesn’t detach from the terminal. It’s recommended not to run this as the root user for security. Otherwise you can add a cron task for the current user with: -

crontab -e

and add the following entry: -

*/30 * * * * cd /var/www/html/tt-rss && /usr/bin/php /var/www/html/tt-rss/update.php -feeds >/dev/null 2>&1

Either way, your feeds will update every 30 minutes.

Okay, so now we need to import the Google Reader data. Login with the default username and password as outlined above. Once you’re logged in, click on Actions in the top right of the screen and then Preferences. Click on the Feeds tab and open the OPML button on the lower part of the screen. Choose the “subscriptions.xml” file you unzipped from the Google Takeout zip file – it’s under the “Reader” sub-directory.

Once you’ve imported your feeds from Google Reader you can now log in to your own server and read your RSS new like normal! Personally, I read a lot of Google Reader stuff on my Android phone, so let’s install the Android app next.

Click here to download and install the Android app for tt-rss. It’s a 7 day trial but the full version is only like £1.72. There are free versions available which you can find if you search the Play store for “ttrss” but I found them buggy. This works a charm.

Once installed, go to the Tiny Tiny RSS app and go to Settings and enter the following information: -

Login: your_tt-rss login - defaults to "admin".
Password: defaults to "password"
Tiny Tiny RSS URL: http://your_ip_address/tt-rss/

If you’re using HTTPS/SSL you’ll need to check the following if you’re using a self-signed SSL certificate.

Accept Any Certificate: Check
No Hostname Verification: Check

The app also has the ability to authenticate Apache Basic Authentication, which if you’re feeling paranoid can be set up here. Back out of this menu and refresh and your Google Reader feeds should start displaying.

Congrats, you now have the exact same Google Reader functionality from your own server :-) Take that Google!

I wonder what they’re going to replace Google Reader with? Something within Google Plus would be my bet.

Update 16/05/2013: You may get a 500 Internal Error under apache when you first start. The two most common reasons for this are your MySQL database is corrupted so delete and reimport your feeds or you don’t have the mbstrings PHP module. If you’re not using a version of PHP that you compiled yourself, you need to download the php-mbstring package. Under CentOS/Fedora, that is “yum install php-mbstring”.

I have a rather nifty screensaver under Ubuntu Linux which can output in a suitably retro CRT-screen style font an RSS feed. What I wanted to do was to have all my Google Reader feed subscriptions scroll by as my screen saver – an RSS feed of my subscriptions to to RSS feeds, if you will :-D

Fortunately, this is totally possible, although Google calls this RSS feed a Google Reader “bundle”. It’s very easy to set up. First, log in to your Google Reader account and then follow this link: -

http://www.google.com/reader/view/#bundle-creator-page

Google will then prompt you to create a new “bundle” so that you can either share it with your mates or plug it into your program of choice. You have to enter a title and then choose which subscriptions to add to the bundle which you can drag and drop into the displayed box to add. Unfortunately, there is no option to “select all” so you’ll have to add them one by one which is a minor annoyance.
Once you’ve saved this bundle, you have several options – one of which is “Add a link” to your website. This is your bundle’s RSS feed URL. Copy this link and you’re set to add it to whatever program you want – including the screen saver I wanted to use. Nice :-D

PodGrab, my Python open source command-line downloader has been updated to version 1.1.0. This will probably be the last version aside from minor bug fixes. New in this version is import/export support for OPML files and more media formats supported than just MP3. It now supports pretty much every type of downloadable audio/video media podcast format out there, aside from pure text feeds. The full list is MP3, M4V, OGG, FLV, MP4, MPG/MPEG, WMA, WMV and WEBM. It currently uses SQLite for it’s subscription database, but if I get enough requests or I want it for myself I’ll add MySQL database support.

You can grab the new version here.

PodGrab, my open source audio podcast RSS downloader has been updated to version 1.0.3. This is the final release before PodGrab 1.2 comes out. I’m planning to add OPML import/export support for the next release.

Fixes for this version include:-

- Downloading previous episodes of a podcast now works again :-)
- Various error detection routines for incorrectly formatted podcast item dates.
- Disallows you to add a podcast subscription to the database if it already exists.
- When you delete a subscription, the files are also downloaded from the podcast download directory.
- Download directory now defaults to ‘podcasts’ rather than ‘Podcasts’ as this was annoying with Linux tab auto-complete.

You can download the latest version here.

PodGrab, my open source command line RSS Podcast downloader, has been updated to version 1.0.2. This update fixes an issue with RSS feeds that are not working properly and recovers from this error gracefully. If you’ve downloaded it before, please get the new version here

PodGrab, my open source command line RSS Podcast downloader, has been updated to version 1.0.1. This update simply fixes a few bugs. If you’ve downloaded it before, please get the new version here

I’ve been using gPodder as a podcast downloader for quite a while, which is great for a desktop system like Ubuntu, but less awesome (i.e. can’t be done) for cron-jobbing on my server. What I wanted was something I could use to store my podcast subscriptions and set it up as a cron job on my server and download all my podcasts to my server whenever there was a new episode out. I had a look around on the net for an RSS-based podcast downloader for the command line and couldn’t find one. I mean, not one.
Since I’ve been meaning to learn Python for quite a while, I thought this would be the ideal opportunity to learn a bit of Python and get a useful tool out of it that I could actually use. So, without further ado, I present to you PodGrab, my Python podcast downloader.

It is very simple to use although if you’re using Windows, you may need to download some extra modules – but since this isn’t a Windows site we’ll forget about that :)

Once you’ve downloaded it, you can add a new feed URL with: -


PodGrab.py -s http://some.feed.url.xml


This will store the feed as a subscription and download the latest episode. Sometimes (as I have wanted to do in the past), you simply want to download all episodes of a particular podcast without subscribing to it. You can do this with: -


PodGrab.py -d http://some.feed.url.xml


If you want to list your current subscriptions, you can use: -


PodGrab.py -l


which will list all your current subscriptions. You can delete a subscription with: -


PodGrab.py -un http://some.feed.url.xml


The program also has the ability to mail you if you add an e-mail address, list mail addresses and such. Type


PodGrab.py -h


for a full list of command line switches. To update your podcast subscriptions when using in conjunction with cron, you simply use: -


PodGrab.py -u


and the program will check each feed for a new episode. By default, all podcast downloads are stored in a subdirectory called “Podcasts” off wherever you run PodGrab for the first time, but this is easily changed in the code if you desire. The subscriptions are all stored in a file called “PodGrab.db” which is a small SQLite database. In a future version, I may add MySQL support.

Anyway, you can download PodGrab here. I hope it’s useful to somebody. If you find any major bugs or suggestions for improvements, my e-mail address is in the code. I hope you enjoy it :-)