Python API: Core Bibolamazi Module

Module contents

Core bibolamazi module.

See bibolamazi.core.bibfilter, bibolamazi.core.bibolamazifile, bibolamazi.core.bibusercache for the main core modules.

Subpackages

• bibolamazi.core.bibfilter package
• bibolamazi.core.bibusercache package

bibolamazi.core.argparseactions module

This module defines callbacks and actions for parsing the command-line arguments for bibolamazi. You’re most probably not interested in this API. (Not mentioning that it might change if I feel the need for it.)

bibolamazi.core.argparseactions.help_list_filters()

bibolamazi.core.argparseactions.helptext_prolog()

class bibolamazi.core.argparseactions.opt_action_help(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class bibolamazi.core.argparseactions.opt_action_version(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class bibolamazi.core.argparseactions.opt_init_empty_template(nargs=1, **kwargs)

Bases: argparse.Action

class bibolamazi.core.argparseactions.opt_list_filters(nargs=0, **kwargs)

Bases: argparse.Action

class bibolamazi.core.argparseactions.opt_set_fine_log_levels(nargs=1, **kwargs)

Bases: argparse.Action

class bibolamazi.core.argparseactions.opt_set_verbosity(nargs=1, **kwargs)

Bases: argparse.Action

bibolamazi.core.argparseactions.run_pager(text)

Call pydoc.pager() in a unicode-safe way.

class bibolamazi.core.argparseactions.store_key_bool(option_strings, dest, nargs=1, const=True, exception=<type 'exceptions.ValueError'>, **kwargs)

Bases: argparse.Action

Handles an ghostscript-style option of the type ‘-dBoolKey’ or ‘-dBoolKey=0’.

class bibolamazi.core.argparseactions.store_key_const(option_strings, dest, nargs=1, const=True, **kwargs)

Bases: argparse.Action

class bibolamazi.core.argparseactions.store_key_val(option_strings, dest, nargs=1, exception=<type 'exceptions.ValueError'>, **kwargs)

Bases: argparse.Action

Handles an ghostscript-style option of the type ‘-sBoolKey=some-value’.

class bibolamazi.core.argparseactions.store_or_count(option_strings, dest, nargs='?', **kwargs)

Bases: argparse.Action

bibolamazi.core.bibolamazifile module

The Main bibolamazifile module: this contains the BibolamaziFile class definition.

bibolamazi.core.bibolamazifile.AFTER_CONFIG_TEXT = "%\n%\n% ALL CHANGES BEYOND THIS POINT WILL BE LOST NEXT TIME BIBOLAMAZI IS RUN.\n%\n\n\n%\n% This file was generated by BIBOLAMAZI __BIBOLAMAZI_VERSION__ on __DATETIME_NOW__\n%\n% https://github.com/phfaist/bibolamazi\n%\n% Bibolamazi collects bib entries from the sources listed in the configuration section\n% above, and merges them all into this file while applying the defined filters with\n% the given options. Your sources will not be altered.\n%\n% Any entries ABOVE the configuration section will be preserved as is, which means that\n% if you don't want to install bibolamazi or if it not installed, and you want to add\n% a bibliographic entry to this file, add it AT THE TOP OF THIS FILE.\n%\n%\n\n\n\n\n\n"

Some text which is inserted immediately after the config section when saving bibolamazi files. Includes a warning about losing any changes.

bibolamazi.core.bibolamazifile.BIBOLAMAZIFILE_INIT = 0

Bibolamazi file load state: freshly initialized, no data read. See doc for BibolamaziFile.

bibolamazi.core.bibolamazifile.BIBOLAMAZIFILE_LOADED = 3

Bibolamazi file load state: data read and parsed, filters instanciated and data from sources loaded. See doc for BibolamaziFile.

bibolamazi.core.bibolamazifile.BIBOLAMAZIFILE_PARSED = 2

Bibolamazi file load state: data read and parsed, filters instanciated but no sources loaded. See doc for BibolamaziFile.

bibolamazi.core.bibolamazifile.BIBOLAMAZIFILE_READ = 1

Bibolamazi file load state: data read, not parsed. See doc for BibolamaziFile.

bibolamazi.core.bibolamazifile.BIBOLAMAZI_FILE_ENCODING = 'utf-8'

The encoding used to read and write bibolamazi files. Don’t change this.

class bibolamazi.core.bibolamazifile.BibolamaziFile(fname=None, create=False, load_to_state=3, use_cache=True, default_cache_invalidation_time=None)

Bases: object

Represents a Bibolamazi file.

This class provides an API to read and parse bibolamazi files, as well as load data defined in its configuration section and interact with its filters.

A BibolamaziFile object may be in different load states:

• BIBOLAMAZIFILE_INIT: The BibolamaziFile object is initialized to an empty state. The file name (fname()) may be set already, but is None by default.
• BIBOLAMAZIFILE_READ: Data has been read from a given file, but not parsed. You may call certain methods such as rawHeader() or configData(), but e.g. configCmds() will return an invalid value.
• BIBOLAMAZIFILE_PARSED: Data has been read from a bibolamazi file and parsed, and filter objects have been instanciated. Methods such as filters() or sourceLists() may be called.
• BIBOLAMAZIFILE_LOADED: The bibolamazi file has been read and parsed, filter objects have been instanciated and bibtex data from the sources has been loaded. This is the “maximally loaded” state.

You may query the load state with getLoadState() and load a bibolamazi file either from the constructor or by calling explicitly load(). Some methods on this object may only be called if the object has reached a certain load state. These methods are documented as such.

The bibliography database is accessed with bibliographyData(). You may change the entries in the database via direct access (using the pybtex API), or using the method setEntries().

To create a new bibolamazi file template, you may specify create=True to the constructor with a valid file name, and save the object.

Create a BibolamaziFile object.

If fname is provided, the file is fully loaded (unless create is also provided).

If create is given and set to True, then an empty template is loaded and the internal file name is set to fname. The internal state will be set to BIBOLAMAZIFILE_LOADED and calling saveToFile() will result in writing this template to the file fname.

If load_to_state is given, then the file is only loaded up to the given state. See load() for more details. The state should be one of BIBOLAMAZIFILE_INIT, BIBOLAMAZIFILE_READ, BIBOLAMAZIFILE_PARSED or BIBOLAMAZIFILE_LOADED.

If use_cache is True (default), then when loading this file, we’ll attempt to load a corresponding cache file if it exists. Note that even if use_cache is False, then cache will still be written when calling saveToFile().

If default_cache_invalidation_time is given, then the default cache invalidation time is set before loading the cache.

bibliographyData()

Return the pybtex.database.BibliographyData object which stores all the bibliography entries.

This object is only instanciated and initialized once in the BIBOLAMAZIFILE_LOADED state. If getLoadState() != BIBOLAMAZIFILE_LOADED, then this function returns None.

bibliographydata()

Deprecated since version 2.0: Use bibliographyData() instead!

cacheAccessor(klass)

Returns the cache accessor instance corresponding to the given class.

See documentation of bibolamazi.core.bibusercache for more information about the bibolamazi cache.

cacheFileName()

The file name where the cache will be stored. You don’t need to access this directly, the cache will be loaded and saved automatically.

Filters should only access the cache through cache accessors. See cacheAccessor().

configCmds()

Return a list of parsed commands from the configuration section.

This returns a list of BibolamaziFileCmd objects.

This may be called in the state BIBOLAMAZIFILE_PARSED.

configData()

Returns the configuration commands, with leading percent signs stripped, and without the begin and end tags.

This may be called in the state BIBOLAMAZIFILE_READ.

configLineNo(filelineno)

Utility to convert file line number to config line number

Returns the line number in the config data corresponding to line filelineno in the file. Opposite of fileLineNo().

This may be called in the state BIBOLAMAZIFILE_READ.

fdir()

Returns the directory name in which this bibolamazi file resides, always as a full path (using os.path.realpath, resolving symlinks). The value is cached, so you may call this function several times with little performance overhead.

If fname() is None (this is only possible if the load state is BIBOLAMAZIFILE_INIT), then None is returned.

fileLineNo(configlineno)

Utility to convert config line number to file line number

Returns the line number in the bibolamazi file corresponding to the config line number configlineno. The configlineno refers to the line number INSIDE the config section, where line number 1 is right after the begin config tag CONFIG_BEGIN_TAG.

This may be called in the state BIBOLAMAZIFILE_READ.

filters()

Return a list of filter instances

This returns the list of all filter commands given in the bibolamazi config section. The instances have already been instanciated with the proper options. The order of this list is exactly the order of the filters in the config section.

If in the config section the same filter is invoked several times, then separate instances are returned in this list with the appropriate ordering, as you’d expect.

fname()

Returns the file name this object refers to.

If the state is any other than BIBOLAMAZIFILE_INIT, then this function will never return None.

getLoadState()

Returns the state of the BibolamaziFile object. One of BIBOLAMAZIFILE_INIT, BIBOLAMAZIFILE_READ, BIBOLAMAZIFILE_PARSED, or BIBOLAMAZIFILE_LOADED.

load(fname=[], to_state=3)

Load the given file into the current object.

If fname is None, then reset the object to an empty state. If fname is not given or an empty list, then use any previously loaded fname and its state.

This function may be called several times with different states to incrementally load the file, for example:

bibolamazifile.reset()
# load up to 'parsed' state
bibolamazifile.load(fname="somefile.bibolamazi.bib", to_state=BIBOLAMAZIFILE_PARSED)
# continue loading up to fully 'loaded' state
bibolamazifile.load(fname="somefile.bibolamazi.bib", to_state=BIBOLAMAZIFILE_LOADED)

If to_state is given, will only attempt to load the file up to that state. This can be useful, e.g., in a config editor which needs to parse the sections of the file but does not need to worry about syntax errors. The state should be one of BIBOLAMAZIFILE_INIT, BIBOLAMAZIFILE_READ, BIBOLAMAZIFILE_PARSED or BIBOLAMAZIFILE_LOADED.

rawConfig()

Return the raw configuration section. The returned value will NOT have the leading percent signs removed.

This may be called in the state BIBOLAMAZIFILE_READ.

rawHeader()

Return any content above the configuration section.

This may be called in the state BIBOLAMAZIFILE_READ.

rawRest()

Return all the contents after the config section at the moment the file was read from the disk. This includes the begin and end config section tags (CONFIG_BEGIN_TAG and CONFIG_END_TAG).

Any changes to the bibliography data will not be reflected here, even if you call saveToFile().

This may be called in the state BIBOLAMAZIFILE_READ.

rawStartConfigDataLineNo()

Returns the line number on which the begin config tag CONFIG_BEGIN_TAG is located. Line numbers start at 1 at the top of the file like in any reasonable editor.

This may be called in the state BIBOLAMAZIFILE_READ.

reset()

Reset the current object to an empty state and unset the file name. This will reset the object to the state BIBOLAMAZIFILE_INIT.

resolveSourcePath(path)

Resolves a path (for example corresponding to a source file) to an absolute file location.

This function expands ‘~/foo/bar’ to e.g. ‘/home/someone/foo/bar’; it also expands shell variables, e.g. ‘$HOME/foo/bar’ or ‘${MYBIBDIR}/foo/bar.bib’.

If the path is relative, it is made absolute by interpreting it as relative to the location of this bibolamazi file (see fdir()).

Note: path should not be an URL.

saveToFile()

Save the current bibolamazi file object to disk.

This will write to the file fname() in order:

• the raw header data (rawHeader()) unchanged
• the config section text (rawConfig()) unchanged
• the bibliography data contained in bibliographyData(), saved in BibTeX format.

A warning message is included after the config section that the remainder of the file was automatically generated.

As the file fname is expected to already exist, it is always silently overwritten (so be careful).

setBibliographyData(bibliographydata)

Set the bibliographydata database object directly.

The object bibliographydata should be of instance pybtex.database.BibliographyData.

Warning

Filters should NOT set a different bibliographydata object: caches might have kept a pointer to this object (see, for example EntryFieldsTokenChecker). Please use setEntries() instead.

setConfigData(configdata)

Store the given data configdata in memory as the configuration section of this file.

This function cleanifies the configdata a bit by adding leading percent signs and forcing a final newline, adds the config section begin and end tags, and then directly calls setRawConfig().

setDefaultCacheInvalidationTime(time_delta)

A timedelta object giving the amount of time for which data in cache is consdered valid (by default).

Note that this function should be called BEFORE the data is loaded. If you just call, for example the default constructor, this might be too late already. If you use the load() function, set the default cache invalidation time before you load up to the state BIBOLAMAZIFILE_LOADED.

Note that you may also use the option in the constructor default_cache_invalidation_time, which has the same effect as this funtion called at the appropriate time.

setEntries(bibentries)

Replace all the entries in the current bibliographydata object by the given entries.

Arguments:

• bibentries: the new entries to set. bibentries should be an iterable of (key, entry) (or, more precisely, any valid argument for pybtex.database.BibliographyData.add_entries()).

Warning

This will remove any existing entries in the database.

This function alters the current bibliographyData() object, and does not replace it by a new object. (I.e., if you kept a reference to the bibliographyData() object, the reference is still valid after calling this function.)

setRawConfig(configblock)

Store the given configblock in memory as the raw configuration section of the bibolamazi file. We must be at least in state BIBOLAMAZIFILE_READ to call this function; this function will also reset to state back to BIBOLAMAZIFILE_READ (as the configuration might have changed).

Note that configblock is expected to start and end with the appropriate config section tags (CONFIG_BEGIN_TAG and CONFIG_END_TAG).

After calling this function, configData() will return the new configuration data. Call load() to re-instanciate filters and re-load sources.

sourceLists()

Return a list of source lists, in the order they are specified in the configuration section.

Each item in the returned list is itself a list of alternative sources to consider.

This may be called in the state BIBOLAMAZIFILE_PARSED.

sources()

Return a list of sources which have been read.

This is a list of strings. Each item in the returned list is one of the items in the corresponding list from sourceLists() (the one that was actually found and read). If no corresponding item in sourceLists() was readable, then the corresponding item in this list is None. For example:

# suppose that we have the following instructions in the bibolamazi file:
#
#     src: src1.bib
#     src: a.bib b.bib c.bib
#     src: x/x.bib y/y.bib
#
# we would then have:
#
f.sourceLists() == [["src1.bib"], ["a.bib", "b.bib", "c.bib"], ["x/x.bib", "y/y.bib"]]

# suppose that "src1.bib" exists, "a.bib" doesn't exist but "b.bib" exists, and neither
# "x/x.bib" nor "y/y.bib" don't exist.
#
# Then, after loading this object, we get:
#
f.sources() == ["src1.bib", "b.bib", None]

This function may be called in the state BIBOLAMAZIFILE_LOADED.

class bibolamazi.core.bibolamazifile.BibolamaziFileCmd(cmd=None, text='', lineno=-1, linenoend=-1, info={})

A command in the bibolamazi file configuration

Stores the command name (e.g. ‘src’ or ‘filter’), additional text (the options), line number information and possible additional information.

Object Properties:

• cmd: the command name. Currently this is ‘src’ or ‘filter’
• text: the text following the command. This is e.g. the sources list, or a filter name followed by options. In general, it is anything following the ‘src:’ or ‘filter:’ commands.
• lineno: the line number at which this command is specified in the bibolamazi file, relative to the top of the file. The first line of the file is line number 1.
• linenoend: the line number at which the command ends.
• info: a dictionary with possible additional information which is available at parse time. For example, the filter name for ‘filter’ commands is stored when parsing commands.

See also bibolamazifile.configCmds().

Construct a BibolamaziFileCmd with the given cmd, text, lineno, linenoend and info.

exception bibolamazi.core.bibolamazifile.BibolamaziFileParseError(msg, fname=None, lineno=None)

Bases: bibolamazi.core.butils.BibolamaziError

bibolamazi.core.bibolamazifile.CONFIG_BEGIN_TAG = '%%%-BIB-OLA-MAZI-BEGIN-%%%'

The line which defines the beginning of a config section in a bibolamazi file.

bibolamazi.core.bibolamazifile.CONFIG_END_TAG = '%%%-BIB-OLA-MAZI-END-%%%'

The line which defines the end of a config section in a bibolamazi file.

exception bibolamazi.core.bibolamazifile.NotBibolamaziFileError(msg, fname=None, lineno=None)

Bases: bibolamazi.core.bibolamazifile.BibolamaziFileParseError

This error is raised to signify that the file specified is not a bibolamazi file—most probably, it does not contain a valid configuration section.

bibolamazi.core.blogger module

Set up a logging framework for logging debug, information, warning and error messages.

Modules should get their logger using Python’s standard logging mechanism:

import logging
logger = logging.getLogger(__name__)

This allows for the user to be rather specific about which type of messages she/he would like to see.

class bibolamazi.core.blogger.BibolamaziConsoleFormatter(ttycolors=False, show_pos_info_level=None, **kwargs)

Bases: logging.Formatter

Format log messages for console output. Customized for bibolamazi.

format(record)

setShowPosInfoLevel(level)

class bibolamazi.core.blogger.BibolamaziLogger(name, level=0)

Bases: logging.Logger

A Logger used in Bibolamazi.

This logger class knows about an additional log level, LONGDEBUG.

Initialize the logger with a name and an optional level.

getSelfLevel()

Returns the level that was set on this logger. If no specific level was set, then returns logging.NOTSET. In this respect, this is NOT the same as getEffectiveLevel().

longdebug(msg, *args, **kwargs)

Produce a log message at level LONGDEBUG.

class bibolamazi.core.blogger.ConditionalFormatter(defaultfmt=None, datefmt=None, **kwargs)

Bases: logging.Formatter

A formatter class.

Very much like logging.Formatter, except that different formats can be specified for different log levels.

Specify the different formats to the constructor with keyword arguments. E.g.:

ConditionalFormatter('%(message)s',
                     DEBUG='DEBUG: %(message)s',
                     INFO='just some info... %(message)s')

This will use ‘%(message)s’ as format for all messages except with level other thand DEBUG or INFO, for which their respective formats are used.

do_format(record, fmt)

format(record)

bibolamazi.core.blogger.logger = <bibolamazi.core.blogger.BibolamaziLogger object>

(OBSOLETE) The main logger object. This is a logging.Logger object.

Deprecated since version 2.1: This object is still here to keep old code functioning. New code should use the following idiom somewhere at the top of their module:

import logging
logger = logging.getLogger(__name__)

(Just make sure the logging mechanism has been set up correctly already, see doc for blogger module.)

This object has an additional method longdebug() (which behaves similarly to debug()), for logging long debug output such as dumping the database during intermediate steps, etc. This corresponds to bibolamazi command-line verbosity level 3.

bibolamazi.core.blogger.setup_simple_console_logging(logger=<logging.RootLogger object>, stream=<open file '<stderr>', mode 'w'>)

Sets up the given logger object for simple console output.

The main program module may for example invoke this function on the root logger to provide a basic logging mechanism.

bibolamazi.core.butils module

Various utilities for use within all of the Bibolamazi Project.

exception bibolamazi.core.butils.BibolamaziError(msg, where=None)

Bases: exceptions.Exception

Root bibolamazi error exception.

See also BibFilterError and BibUserCacheError.

bibolamazi.core.butils.call_with_args(fn, *args, **kwargs)

Utility to call a function fn with *args and **kwargs.

fn(*args) must be an acceptable function call; beyond that, additional keyword arguments which the function accepts will be provided from **kwargs.

This function is meant to be essentially fn(*args, **kwargs), but without raising an error if there are arguments in kwargs which the function doesn’t accept (in which case, those arguments are ignored).

bibolamazi.core.butils.get_version()

Return the version string version_str, unchanged.

bibolamazi.core.butils.get_version_split()

Return a 4-tuple (maj, min, rel, suffix) resulting from parsing the version obtained via version.version_str.

............ TODO: FIXME: CURRENTLY, the elements are strings! why not integers? If not there, they will/should be empty or None?

bibolamazi.core.butils.getbool(x)

Utility to parse a string representing a boolean value.

If x is already of integer or boolean type (actually, anything castable to an integer), then the corresponding boolean convertion is returned. If it is a string-like type, then it is matched against something that looks like ‘t(rue)?’, ‘1’, ‘y(es)?’ or ‘on’ (ignoring case), or against something that looks like ‘f(alse)?’, ‘0’, ‘n(o)?’ or ‘off’ (also ignoring case). Leading or trailing whitespace is ignored. If the string cannot be parsed, a ValueError is raised.

bibolamazi.core.butils.guess_encoding_decode(dat, encoding=None)

bibolamazi.core.butils.parse_timedelta(in_s)

Note: only positive timedelta accepted.

bibolamazi.core.butils.quotearg(x)

bibolamazi.core.butils.resolve_type(typename, in_module=None)

Returns a type object corresponding to the given type name typename, given as a string.

..... TODO: MORE DOC .........

bibolamazi.core.butils.warn_deprecated(classname, oldname, newname, modulename=None, explanation=None)

bibolamazi.core.main module

This module contains the code that implements Bibolamazi’s main functionality. It also provides the basic tools for the command-line interface.

class bibolamazi.core.main.AddFilterPackageAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class bibolamazi.core.main.ArgsStruct(bibolamazifile, use_cache, cache_timeout)

Bases: tuple

bibolamazifile

Alias for field number 0

cache_timeout

Alias for field number 2

use_cache

Alias for field number 1

exception bibolamazi.core.main.BibolamaziNoSourceEntriesError

Bases: bibolamazi.core.butils.BibolamaziError

bibolamazi.core.main.get_args_parser()

bibolamazi.core.main.main(argv=['-T', '-b', 'readthedocsdirhtml', '-D', 'language=en', '.', '_build/html'])

bibolamazi.core.main.run_bibolamazi(bibolamazifile, **kwargs)

bibolamazi.core.main.run_bibolamazi_args(args)

bibolamazi.core.main.setup_filterpackage_from_argstr(argstr)

Add a filter package definition and path to filterfactory.filterpath from a string that is a e.g. a command-line argument to –filterpath or a part of the environment variable BIBOLAMAZI_FILTER_PATH.

bibolamazi.core.main.setup_filterpackages_from_env()

bibolamazi.core.main.verbosity_logger_level(verbosity)

Simple mapping of ‘verbosity level’ (used, for example for command line options) to correspondig logging level (logging.DEBUG, logging.ERROR, etc.).

bibolamazi.core.version module

bibolamazi.core.version.version_str = '3.0'

The version string. This is increased upon each release.