Installing AceDB locally.

[Paulie]

Hi all (first post - be gentle :D)

I am trying to install AceDB locally - on a machine where I am not root, but
I can compile and run programmes and I have MySQL running on this machine and
can create and install databases. I can also either install or request to have installed
libraries.

My understanding is that AceDB is the core of Wormbase with MySQL databases
“hanging off” the AceDB core. I compiled and ran AceDB with the demo (see below
for how)

I went to ftp://ftp.sanger.ac.uk/pub/wormbase/releases/WS239/acedb/ and downloaded
the datafiles (well, a couple) checked them and found that some were binaries after tar -zxvf.

My understanding is that (some of) these are dumps of MySQL databases for each species present in
the data files. What I would like are instructions on how to load what and where.

I would also like to run the full AceDB database/system (which doesn’t appear to be included
in the distro?) against these MySQL databases. Exhaustive seaching has revealed this

http://wiki.wormbase.org/index.php/Administration:Installing_WormBase#MySQL

But the critical section contains this:
Installing Databases NOT DONE Primary database (AceDB)

Can anyone point me to a document telling what to do with what files so that
I may start to use AceDB?

Any other useful material appreciated.

Rgs,

Paul…

========== Compiling and running AceDB =============

I downloaded the latest monthly build of AceDB, ran make (ACEDB_MACHINE=LINUX_64)
and that went fine - a few warnings. Got bin.LINUX_64 in my acedb directory.

Ran xace from the command line and was told that I would have to run where there
was a database and wspec file. Searched for these and found them in wdemo - ran
xace …/wdemo and was warned about a password file - put my username in there
and was asked if I’d like to create the database. I said yes and was able to run
the interface against the sample.

[mh6]

a.) you want a working ACeDB program, a Linux version you can get from http://www.acedb.org/ (it has X11 and GTK2 dependencies).
If you don’t have a linux box (a.e. a Mac), and you want to compile it, it needs the above dependencies and compiles best with gcc

b.) the tarball of each wormbase release (the .tar.gz files from a.e. ftp://ftp.wormbase.org/pub/wormbase/releases/WS239/acedb/)

These tarball unpacks into a database, wspec and wgf directory. The database contains the binary database files, the wspec the configuration files and the wgf the data needed for the build in splice site and coding potential prediction.

If you start the GUI of acedb (xace) in the directory above the database/wspec/wgf directory, you will get a window that lets you query for things, edit and display (from cell fate trees to genes and genetic maps).
But as the ACeDB database (which was created a few decades back) is slowly creaking with 20+ GB databases, some queries will take a few seconds/minutes (which is quite often too slow to drive a website).
On the positive side the query interface lets you build quite sophisticated queries that are not easily replicated in relational schemas, plus the CDS editing interface is still one of the best I have used so far (but struggles a bit with RNASeq data … but at least it shows it).

And we currently do not provide mysql dumps, but rather make data available as fasta and GFF2/3 files, as they are more universally usable.

[Paulie]

Hi, and thanks for answering - and please forgive in advance my ignorance.

@mh6 post:2

a.) you want a working ACeDB program, a Linux version you can get from http://www.acedb.org/ (it has X11 and GTK2 dependencies).
If you don’t have a linux box (a.e. a Mac), and you want to compile it, it needs the above dependencies and compiles best with gcc

As I said originally, I have the programme compiled and apparently working against the demo in
the wdemo subdirectory.

@mh6 post:2

b.) the tarball of each wormbase release (the .tar.gz files from a.e. ftp://ftp.wormbase.org/pub/wormbase/releases/WS239/acedb/)

OK - that’s fine - I have downloaded a couple and unzipped/untarred them. The .tar.gz come
in directories 0 - 8 - I created directories data0 and data1 for my test and downloaded
and tar -zxvf’ed them in those directories.

Do I have to organise these directories some way? See below - I think I’ve resolved this.

@mh6 post:2

These tarball unpacks into a database, wspec and wgf directory. The database contains the binary
database files, the wspec the configuration files and the wgf the data needed for the build in
splice site and coding potential prediction.

OK, fine - that much I can readily grasp!

@mh6 post:2

If you start the GUI of acedb (xace) in the directory above the database/wspec/wgf directory, you
will get a window that lets you query for things, edit and display (from cell fate trees to
genes and genetic maps).

I did this in a directory data0 into which I had unzipped the file database.239.4-0.tar.gz. This produced
subdirectories database, wgf, wquery and wspec.

I then ran path_to_xace/xace . // dot to specify this directory and was told that that it could
not read block1.wrm and that it had to exit. I then realised that in the first datafile there were
blocks1…5 and in the second files6…10 - and I presume &c… I will download all of the datafiles
and set run xace when I have them all. I will report back on this.

@mh6 post:2

But as the ACeDB database (which was created a few decades back) is slowly creaking with 20+ GB databases,
some queries will take a few seconds/minutes (which is quite often too slow to drive a website).

OK, that’s fine - I understand that not every system can be running Oracle Enterprise server with
Node.js serving us all-singing all-dancing graphics of genes recombining in real time (or whatever
your fancy system/interfaces are doing this week).

My question would then be, how is it that AceDB is still part of (AIUI) the core of Wormbase
(which I eventually hope to get up and running)? The install Wormbase site I originally
referred to makes specific reference to this.

@mh6 post:2

On the positive side the query interface lets you build quite sophisticated queries that are not easily replicated
in relational schemas, plus the CDS editing interface is still one of the best I have used so far (but struggles
a bit with RNASeq data … but at least it shows it).

Great - this again I get!

@mh6 post:2

And we currently do not provide mysql dumps, but rather make data available as fasta and GFF2/3 files, as they are more universally usable.

I have seen material about loading up MySQL databases for various species -

From Administration:Installing WormBase - WormBaseWiki

============================
Set up mysql permissions

$ mysql -u root -pPASSWORD
mysql> grant select on c_elegans.* to nobody@localhost;

Repeat for:

autocomplete
b_malayi
c_briggsae 

==========================================

Where can I get the data for these? And what is the autocomplete data/database?

Are the data here: ftp://ftp.wormbase.org/pub/wormbase/releases/WS239/species/
suitable for inputting into a database? Or do I have to write scipts for myself?

Are the block1…blockn.wrm directories duplicate data? Or are the fasta/gff data
duplicates of the block data?

Thanks for your input so far.

Rgs,

Paul…

[mh6]

you need to untar the files, so the result looks like:
ftp://ftp.wormbase.org/pub/wormbase/releases/WS239/acedb/files_in_tar
in theory you can just untar each tarfile in the same directory and it should put the files in the correct sub-directories.
as in cd your_current_working_directory

and then assuming you use tcsh (if you use some other shell, just change the foreach to the respective shell variant)

foreach i(*tar.gz))
tar xvfz $i
end

so you should end up with something that looks like:

your_current_working_directory/
database <= that is the one with the block files
wspec <= that is where the configuration files reside
wgf <= hexamer / splice site matrices
wquery <= precanned queries (optional)

then cd into your_current_working_directory and run xace .
and you should be fine.

[Paulie]

Hi and thanks again for your original post, it set me on the right path.

@mh6 post:4

foreach i(*tar.gz))
tar xvfz $i
end

I created a shell script creating symbolic links to my downloads in the directory
you mentioned. The final size of the database is ~ 35GB by my reckoning. It’s
pretty slow starting up, but seems to be fairly nippy on queries.

I managed to configure the saceserver and can run the saceclient against it.

One small worry though, it throws a warning (sh: 1: uuencode: not found) when
I start saceserver - is this to be ignored or is it important?

I’m going to start a new thread about obtaining the MySQL data

Thanks again and rgs,

Paul…

Rgs,

Paul…