Richard Martin's Blog

I’m a great fan of Bucardo for PostgreSQL replication…. it’s stable but quite hard to use…. if you want fire and (not) forget replication try RubyRep…. awesome.  The documentation is perfect but to make it even easier, if you are testing and messing around… uninstall, then re-scan, then replicate otherwise you might get some bizarre replication errors, or foreign key contraint issues etc… anything odd, just un-install, then re-scan and replicate.  It really is full proof… well it worked for me!

For the cut & paste brigade: (RTFM)

rubyrep generate myrubyrep.conf

rubyrep scan -c myrubyrep.conf

nohup rubyrep replicate -c myrubyrep.conf &

Remember the nohup and the &, to background it, unless you’re actually at the console of the production server – which you probably aren’t….  In fact stick the nohup and & on the scan command too as that could take a few hours on a big DB…

Then tail -f nohup.out to watch the (lack) of output, output is normally bad – loss of connection:

2009-07-31T22:22:13+01:00 Exception caught: no connection to ‘right’ database

and what not… but rubyrep will just keep going once the connection comes back up.  Perfect!

If you live in the UK and like ‘proper’ dates use this:

PythonOption TracLocale en_GB

in your httpd.conf (between the locations)

I’ve been playing with this for years.  Nothing works without major work – OK I’m lazy.  Now I can spend $500 and get pretty good replication for my linux servers, single windows server, and all the code, stuff and my precious PostgreSQL databases – at last.  I love Bucardo for great FREE replication and now I know how it works I’ll be using all over the place but if you want an easy life and good sleep check out R1 software

Boy, it’s made my life a lot easier…….

unless you don’t backup your CDP linux backup software database….. and then you lose ALL your backups…. so if you use CDP backup software backup your /usr/r1soft directly ALL the time…. if that directory gets corrupted – you’re screwed….

To setup Bucardo on a server as a multi-master

All the code and stuff is on Main server – the other server just listens…. so it does not need any “config”

To install (on the main server):

As root:
yum install postgresql-pl.x86_64
cpan
install Bundle::CPAN

then install the required Perl modules with
install DBI etc…

To install:
as postgres in psql:
CREATE USER bucardo SUPERUSER;

on ALL target databases: (as postgres)
createlang plpgsql DATABASE

as postgres in psql:
CREATE DATABASE bucardo OWNER bucardo;

as bucardo in psql:
CREATE LANGUAGE plpgsql;
CREATE LANGUAGE plperlu;

as postgres:
psql -f bucardo.schema -U bucardo bucardo

To make importable SQL files you can to save you some typing:
(inside psql)
\t
\o filename.sql
RUN A SQL QUERY and the output goes to the filename.sql
you can then just \i the files

RUN THE SQL for INSERTS and LINKS

To setup the replication and get it all going:
as bucardo on the main server:
INSERT INTO db(name, dbname, dbhost, dbuser)
VALUES ('main-server','DB_NAME','nnn.nnn.nnn.nnn','DB_USER');
INSERT INTO db(name, dbname, dbhost, dbuser)
VALUES ('other-server','DB_NAME',' nnn.nnn.nnn.nnn','DB_USER');

INSERT INTO dbgroup(name) VALUES (’remote_dbs’);
INSERT INTO dbmap(db,dbgroup) VALUES (’REMOTE_DB’,'remote_dbs’);

To generate the inserts you need: as user DB_USER

\t
\o tables.sql

This gives you a list of all tables you (probably want to replicate):
select 'INSERT INTO goat(db, tablename, pkey, pkeytype, standard_conflict) VALUES(\'main-server\', \'' || relname || '\', \'id\', \'int\', \'source\');'
from pg_class, pg_user
where usesysid=relowner and
(relkind ='r' or relkind='s') and (usename ='DB_NAME') and
(relkind !='i' or relname !~'^xinx')
order by relname;

\o

** MAKE SURE YOU REMOVE ANY REFERENCES TO BUCARDO ** – I just delete them by hand

then as user BUCARDO

(continue reading…)