How can I (partially) automate bibliography management?

Question

I've amassed a not-huge-but-not-tiny bibliography over the years, which I occasionally use. So far, it has had the forms of several .bib files which drag around from one paper to the next, occasionally getting some more text, or "sacrificed" in order to make long-form rather than short-form entries (and those don't get updated) etc. I've managed / not really tried to get aliases to work, and I have all entries conforming to a certain naming scheme. Also, I add entries manually, including making sure I like their format, removing fields I don't think I need, and even tinkering with spacing .

I would like to make all this more flexible and manageable with some automation: I want to be able to generate .bib files for a specific paper where I specify:

• A citation key format
• Level of verbosity (not just journal name abbreviation)
• Trimming unused entries
• Some kind of smart aliasing without seeing dupes in the bibliography etc.
• Possibly other nifty goodies

Restrictions/requirements:

• Must be gratis and libre software
• Command-line is good, GUI is ok, both of them is best

Note: This is a similar question, it's coming from an MS-Word + Zotero user (of while I am neither)

Answer 1

I'm not sure if I get all of your points, but let me try to answer at least parts of your question.

• It is not a bad idea to have one big .bib file with all works that are of interest to you.

• Choose sensible entry keys for all of your entries that you use in all papers. That makes it easier for you to recognise which entry is which even across papers.

• Include all information in your main .bib file.

  • It is the job of your bibliography style (.bst file or biblatex style) to decide which fields get printed. If you think the output is too wordy, modify the style - don't modify the source. There should be no 'long-form' or 'short-form' of the same entry, there should just be one entry.
  • There may be one exception to that rule: LaTeX's bibliography styles can not automatically convert journals between long and short form for you. Solutions with varying degrees of automation are available (JabRef has a feature for that, you could use @strings, Biber, or one of these tools). These tools then allow you to export only a subset of the entries to a .bib file and may even allow for post-processing of the entries.
  • Do not tinker with spacing in your .bib file. Ad-hoc modifications should be a last resort

You don't even have to write that one huge .bib file by hand, you can use one of the many reference managers available (see LaTeX and bibliography management tools). I must say I prefer to write my .bib files by hand, but then I don't have a huge database yet.

There are several other tools for manipulation of .bib files. BibTool comes to mind, Biber can be used in a --tool mode as well (even though that is not the primary purpose of Biber). See for example Creating .bib file containing only the cited references of a bigger .bib file. People have written and released various tools to manipulate .bib files further.

In the comments it emerged that you are not a fan of JabRef, but at least formally it would tick almost all of your boxes. A possible alternative is Zotero together with https://retorque.re/zotero-better-bibtex/. But you could also look into helpers like https://github.com/nschloe/betterbib, https://github.com/pubs/pubs, https://github.com/rudrab/MkBiB, and https://github.com/bibcure/bibcure. You may not be able to find one software that does all of what you want in one go, but I'm quite confident that you can find an assortment of tools that can at least get you a little bit closer to a nicer solution for your bibliography.

Some programming languages have libraries for parsing .bib files, so in theory you could also code your own little helper.