Category Archives: Scripting

Installing Gerd Neu­ge­bauer’s Bibtool on OS X

I struggled for more hours than I would like to say trying to compile Gerd Neugebauer’s BibTool utility on my mac (OS 10.9.2). I kept running into problems with the kpathsea libraries. Not surprisingly, an old thread on the Mac OS-TeX mailing list provided the necessary insight. I’m reposting it here so that it might get better exposure on the Internet for those trying to do the same thing. Of course, someone who wasn’t shamelessly hacking away at it probably wouldn’t have had any problem at all . . .

BibTool is an amazing utility that does all those things we all wished BibDesk would do, but doesn’t: command line tools for removing fields, rewriting fields, grabbing references based on various criteria including regular expressions, pretty printing.

On Nov 23, 2009, at 1:49 PM, Jan Erik Moström wrote:

> I’m a bit slow (in all kind of ways 😉 and I’m just trying to figure out the MacTeX distribution … one thing that that I would like to do is to install “bibtool” (not bibtools) but I can’t figure out how to do this in a simple way. Using Gerbens TeX distribution it was possible to use the i-installer but now I’m at a loss how to do it.

You can download the source and compile it yourself from
(Get the *.gz) file.
Once you have expanded the tar archive, copy the makefile.unx file to makefile, then run
sudo make install
sudo make
This will place a working copy and manual pages in the /usr/local tree.
As long as you have /usr/local/bin in your path, things should work.

Of course, you have to have the developer tools to do this.
There are many options to set specific details during the compilation process, but I just did a vanilla “make” and
./bibtool -h
gave me the typical “usage” statement.

I’ve never used bibtool, don’t know how to use it, and don’t have any test files to ensure that it works in a real-life situation.
Otherwise, I now have a seemingly-functional version of “bibtool” that runs on X86_64 Snow Leopard.
I don’t guess that really helps you, but if you want it, send me a message off-list.

BibTeX at the Darwin Manuscripts Project and BHL

Poking around the Biodiversity Heritage Library Tools page, I came across this question from the FAQ:

Question: What is the BibTex format that I see as a download option?

BibTex ( is a common format for citations/references and is supported by all the major software vendors (EndNote, RefWorks, Zotero, Biblio). This functionality that lets a user view & export a BibTex file for any title, including its items, from the bibliography page, as in this example:

BHL is also going to make this format available for download alongside our custom data exports, such that users can download a BibTex file
that contains 1) all the *titles* in BHL including links to each, and 2) all the *items* in BHL (each volume) along with links. We need this export to move title-level metadata from the BHL portal to the article repository, so thought we might as well make the file available for others to use.
In effect, this would put BHL titles & volumes in a format easily understood by existing reference management applications.

When deciding whether our big database of works about evolution at the Darwin Manuscripts Project would use Endnote or BibTeX managed by way of BibDesk, I opted for BibTeX—a smart decision, if I do say so myself. It’s served us well in the many years that we’ve been using it, and it looks like it will continue to be useful. Nelson Beebe is developing (or has completed development) on some scripts to represent BibTeX databases in my SQL tables. He provides some useful links to related software tools which are needed as adjuncts to his scripts. In a paper in TUG (forthcoming? in vol 30, issue 1, Nov 2009), he explains a little bit about BibTeX, relational databases, and what’s involved in representing a .bib file as a relational database.

If anyone out there’s had experience creating relational databases from .bib files, feel free to comment on this post, or to let me know how I can contact you to ask questions and listen to any tips, warnings, etc. you might have.

I have to learn a little about Python and mySQL. . .

Lindsay Cowell, who works on the Infectious Disease Ontology, was kind enough to share a script her group used to create a local mySQL database of PubMed records. It’s in Python, and Changqing Li wrote it. In order to use the script, I have to learn about Python and also more about sql databases.  I found a Python tutorial, written by the creator of Python himself! Most likely I will rely on the kindness of my friends to help coach me through the Python scripting, and once it’s in mySQL form, I can take it from there. There’s an issue about where the data is going to go, but Chanqing tells me that the whole thing requires at least 60GB. I hope it doesn’t come to this, but the best (cheapest) course of action might be to create the database on a local disk.  The records I’ll eventually want to incorporate into the Literature of Evolution database will not, I imagine, require so much disk space. I have a few GB’s at geekisp, and maybe the Darwin Manuscripts Project can spare a few.