IT Poetry


Bash script to export and backup online data
February 7, 2008, 11:10 am
Filed under: Internet, Productivity

Even though most sites probably save more data than we’d need for longer than we care (or want). It’s allways nice to keep your own data some place safe. Its also nice to automate backing up your online data, so for your convenience here’s a bash script to back up your online data from del.icio.us, wordpress.com and bloglines.
I’m using the script in linux and cygwin, but it should work with regular bash+wget.
The script works by getting the login page with wget and saving the login cookie. The script asks for your password, but you’re only required to enter it the first time you log in.

updateMyBackups.sh

#!/bin/bash
#Bash script to download your online data, use freely no responsibility taken.
#Author Hugo Hallman, https://itpoetry.wordpress.com/
#License GPL, Please post your updates in comments on the blog.
#USAGE: Either change the function calls at the bottom,
#or write a new script importing this one, calling the export functions.
#EG:
#file: myOnlineBackups.sh
# . updateMyBackups.sh
# noninteractive=0;
# wordPressExport "username" "blogname";
# deliciousExport "username";
# bloglinesExport "me@mail.com";
# end file.

# Set to 1 if you don't want the script to ask questions
noninteractive=0;

function wordPressExport() {
	wpuser=$1;
	wpblog=$2;
	[ "$wpuser" == "" ] && echo "wordpress.com: please give user name" && return 1;
	[ "$wpblog" == "" ] && echo "wordpress.com: please give blog name" && return 1;

	echo "Exporting WordPress: $wpblog by $wpuser";

	mkdir -p temp
	cf=temp/auth_wp;
	touch $cf

	if [ "$noninteractive" != "1" ]; then
		echo "If you want to renew login cookie, type wordpress password: ";
		read wppwd;
		if [ "$wppwd" != "" ] ; then
			echo "Logging in to wordpress.com";
		    wget --keep-session-cookies --save-cookies $cf --quiet --output-document=/dev/null "http://${wpblog}.wordpress.com/wp-login.php";
		    wget --keep-session-cookies --load-cookies $cf --save-cookies $cf --output-document=temp/login.html --quiet \
				--post-data="log=${wpuser}&pwd=${wppwd}&rememberme=on&redirect_to=wp-admin%2F&testcookie=1&wp-submit=Log%20in%20%26raquo%3B" \
				"http://${wpblog}.wordpress.com/wp-login.php";
		fi;
	fi;

	target=${wpblog}.xml;
	out=temp/${target};
	if grep ".wordpress.com" $cf | grep "$wpuser" > /dev/null ; then
		rm -f $out;
		echo "Downloading export to $out";
		wget --quiet --load-cookies $cf --output-document=$out \
			"http://${wpblog}.wordpress.com/wp-admin/export.php?download=true&submit=Download%20Export%20File%20ยป"
		if test -f $out && grep ' /dev/null && grep ' /dev/null; then
			mv $out $target;
			echo "Exported ${target}";
		else
			echo "Failed to export ${target}";
		fi;
	else
		echo "Not logged in to wordpress.com";
	fi;
}

function deliciousExport() {
	user=$1;
	[ "$user" == "" ] && echo "del.icio.us: please give user name" && return 1;

	echo "Exporting del.icio.us tags for $user";

	mkdir -p temp
	cf=temp/auth_delicious;
	touch $cf

	if [ "$noninteractive" != "1" ]; then
		echo "If you want to renew login cookie, type del.icio.us password: ";
		read pwd;
		if [ "$pwd" != "" ] ; then
			echo "Logging in to del.icio.us";
			wget --keep-session-cookies --save-cookies $cf --quiet --no-check-certificate \
				--post-data="user_name=${user}&password=${pwd}&action=login" --output-document=temp/login "https://secure.del.icio.us/login" ;
		fi;
	fi;

	target=delicious.html
	out=temp/${target};
	if test -f $cf && grep ".del.icio.us" $cf | grep "_user" > /dev/null ; then
		rm -f $out;
		echo "Downloading export to $out";
		wget --quiet --no-check-certificate --load-cookies $cf --output-document=${out} \
			--post-data="export=export%20to%20html&showtags=on&showextended=on" \
			"https://secure.del.icio.us/settings/${user}/bookmarks/export";
		if test -f $out && grep '' $out > /dev/null; then
			mv $out $target;
			echo "Exported $target";
		else
			echo "Failed to export $target";
		fi;
	else
		echo "Not logged in to del.icio.us.";
	fi;
}

function bloglinesExport() {
	user=$1;
	[ "$user" == "" ] && echo "bloglines.com please give user name" && return 1;

	echo "Exporting bloglines subscriptions for $user";

	mkdir -p temp
	cf=temp/auth_bloglines;
	touch $cf

	if [ "$noninteractive" != "1" ]; then
		echo "If you want to renew login cookie, type bloglines.com password: ";
		read pwd;
		if [ "$pwd" != "" ] ; then
			echo "Logging in to bloglines.com";
	        wget --keep-session-cookies --save-cookies $cf --quiet --no-check-certificate \
				--post-data="email=${user}&password=${pwd}&signin=Log%20In" \
				--output-document=temp/login \
				"http://www.bloglines.com/login" ;
		fi;
	fi;

	target=blogs.opml;
	out=temp/${target};
	if test -f $cf && grep ".bloglines.com" $cf | grep "BloglinesTracker" > /dev/null ; then
		rm -f $out;
		echo "Downloading export to $out";
		wget --quiet --load-cookies $cf \
			--output-document=$out \
			"http://www.bloglines.com/export";
		if test -f $out && grep ' /dev/null && grep ' /dev/null; then
			mv $out $target;
			echo "Exported $target";
		else
			echo "Failed to export $target";
		fi;
	else
		echo "Not logged in to bloglines";
	fi;
}

#Export from wordpress
wordPressExport "username" "blogname";
wordPressExport "username" "secondblogname";

#Export from del.icio.us
deliciousExport "username";

#Export from bloglines
bloglinesExport "me@mail.com";
Advertisements

6 Comments so far
Leave a comment

Hi, I’m glad it’s not only me who is nerdy enough to need to wget this stuff from a script and svn commit it ๐Ÿ™‚ I like your approach to doing the authentication from the script as opposed to stealing it from the browser like I did. And you backup more services ๐Ÿ™‚

Comment by Adrian Smith

Yes, isn’t it nice. It took some time to figure out the –post-data though, but it’s worth it if its useful to somebody else. (-tell me if you find it usefull)

WordPress is interesting, because they do a cookie-test before letting you log on, so you have to load the start page returning to the log in page to prove that you remember the cookies. Everybody should do something similar, really. (Unless you use something better than PHP.)

Comment by Hugo Hallman

THANKS! Just what i’ve looked for! ๐Ÿ™‚

Comment by Thomas

What about an WordPress-Import? Has anyone did this before?

TIA
Ali

Comment by ali ben hassan

share web script for make SEO

Comment by allshare blog

Great blogpost, I did not thought it was going to be so interesting when I read the title with link!!

Comment by Mulkpeerpet




Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s



%d bloggers like this: