bibolamazi.core.bibfilter package

Module bibolamazi.core.bibfilter

class bibolamazi.core.bibfilter.BibFilter(**kwargs)

Bases: object

Base class for a bibolamazi filter.

To write new filters, you should subclass BibFilter and reimplement the relevant methods. See documentation of the different methods below to understand which to reimplement.

Constructor. Sets the filtername to the name of the filter class.

A warning is emitted for each argument in kwargs, which presumably was ignored by the superclass and passed on.

BIB_FILTER_BIBOLAMAZIFILE = 3

A constant that indicates that the filter should act upon the whole bibliography at once. See documentation for the action() method for more details.

BIB_FILTER_SINGLE_ENTRY = 1

A constant that indicates that the filter should act upon individual entries only. See documentation for the action() method for more details.

action()

Return one of BIB_FILTER_SINGLE_ENTRY or BIB_FILTER_BIBOLAMAZIFILE, which tells how this filter should function. Depending on the return value of this function, either filter_bibentry() or filter_bibolamazifile() will be called.

If the filter wishes to act on individual entries (like the built-in arxiv or url filters), then the subclass should return BibFilter.BIB_FILTER_SINGLE_ENTRY. At the time of filtering the data, the function filter_bibentry() will be called repeatedly for each entry of the database.

If the filter wishes to act on the full database at once (like the built-in duplicates filter), then the subclass should return BIB_FILTER_BIBOLAMAZIFILE. At the time of filtering the data, the function filter_bibolamazifile() will be called once with the full BibolamaziFile object as parameter. Note this is the only way to add or remove entries to or from the database, or to change their order.

Note that when the filter is instantiated by a BibolamaziFile (as is most of the time in practice), then the function bibolamaziFile() will always return a valid object, independently of the filter’s way of acting.

bibolamaziFile()

Get the BibolamaziFile object that we are acting on. (The one previously set by setBibolamaziFile().)

There’s no use overriding this.

cacheAccessor(klass)

A shorthand for calling the cacheAccessor() method of the bibolamazi file returned by bibolamaziFile().

filter_bibentry(x)

The main filter function for filters that filter the data entry by entry.

If the subclass’ action() function returns BibFilter.BIB_FILTER_SINGLE_ENTRY, then the subclass must reimplement this function. Otherwise, this function is never called.

The object x is a pybtex.database.Entry object instance, which should be updated according to the filter’s action and purpose.

The return value of this function is ignored. Subclasses should report warnings and logging through Python’s logging mechanism (see doc of core.blogger) and should raise errors as BibFilterError (preferrably, a subclass). Other raised exceptions will be interpreted as internal errors and will open a debugger.

filter_bibolamazifile(x)

The main filter function for filters that filter the data entry by entry.

If the subclass’ action() function returns BibFilter.BIB_FILTER_SINGLE_ENTRY, then the subclass must reimplement this function. Otherwise, this function is never called.

The object x is a BibolamaziFile object instance, which should be updated according to the filter’s action and purpose.

The return value of this function is ignored. Subclasses should report warnings and logging through Python’s logging mechanism (see doc of core.blogger) and should raise errors as BibFilterError (preferrably, a subclass). Other raised exceptions will be interpreted as internal errors and will open a debugger.

classmethod getHelpAuthor()

Convenience function that returns helpauthor, with whitespace stripped. Use this when getting the contents of the helpauthor text.

There’s no need to (translate: you should not) reimplement this function in your subclass.

classmethod getHelpDescription()

Convenience function that returns helpdescription, with whitespace stripped. Use this when getting the contents of the helpdescription text.

There’s no need to (translate: you should not) reimplement this function in your subclass.

classmethod getHelpText()

Convenience function that returns helptext, with whitespace stripped. Use this when getting the contents of the helptext text.

There’s no need to (translate: you should not) reimplement this function in your subclass.

getRunningMessage()

Return a nice message to display when invoking the fitler. The default implementation returns name(). Define this to whatever you want in your subclass to describe what you’re doing. The core bibolamazi program displays this information to the user as it runs the filter.

helpauthor = ''

Your subclass should provide a helpauthor attribute, containing a one-line notice with the name of the author that wrote the filter code. You may also add a copyright notice. The exact format is not specified. This text is typically displayed at the top of the page generated by bibolamazi --help <filter>.

You should also avoid accessing this class attribute, you should use getHelpAuthor() instead, which will ensure that whitespace is properly stripped.

helpdescription = 'Some filter that filters some entries'

Your subclass should provide a helpdescription attribute, containing a one-line description of what your filter does. This is typically displayed when invoking bibolamazi --list-filters, along with the filter name.

You should also avoid accessing this class attribute, you should use getHelpDescription() instead, which will ensure that whitespace is properly stripped.

helptext = ''

Your subclass should provide a helptext attribute, containing a possibly long, as detailed as possible description of how to use your filter. You don’t need to provide the basic ‘usage’ and option list, which are automatically generated; but you should include all the text that would appear after the option list. This is typically displayed when invoking bibolamazi --help <filter>.

You should also avoid accessing this class attribute, you should use getHelpText() instead, which will ensure that whitespace is properly stripped.

name()

Returns the name of the filter as it was invoked in the bibolamazifile. This might be with, or without, the filterpackage. This information should be only used for reporting purposes and might slightly vary.

If the filter was instantiated manually, and setInvokationName() was not called, then this function returns the class name.

The subclass should not reimplement this function unless it really really really really feels it needs to.

postrun(bibolamazifile)

This function gets called immediately after the filter is run, before any further filters are executed.

It is not very useful if the action() is BibFilter.BIB_FILTER_BIBOLAMAZIFILE, but it can prove useful for filters with action BibFilter.BIB_FILTER_SINGLE_ENTRY, if any sort of global post-processing task should be done immediately after the actual filtering of the data.

You can use this function, e.g., to produce an aggregated warning or report message.

This method is not called if the filter raised an exception, whether internal or not.

The default implementation does nothing.

prerun(bibolamazifile)

This function gets called immediately before the filter is run, after any preceeding filters have been executed.

It is not very useful if the action() is BibFilter.BIB_FILTER_BIBOLAMAZIFILE, but it can prove useful for filters with action BibFilter.BIB_FILTER_SINGLE_ENTRY, if any sort of pre-processing task should be done just before the actual filtering of the data.

The default implementation does nothing.

requested_cache_accessors()

This function should return a list of bibusercache.BibUserCacheAccessor class names of cache objects it would like to use. The relevant caches are then collected from the various filters and automatically instantiated and initialized.

The default implementation of this function returns an empty list. Subclasses should override if they want to access the bibolamazi cache.

setBibolamaziFile(bibolamazifile)

Remembers bibolamazifile as the BibolamaziFile object that we will be acting on.

There’s no use overriding this. When writing filters, there’s also no need calling this explicitly, it’s done in BibolamaziFile.

setInvokationName(filtername)

Called internally by bibolamazifile, so that name() returns the name by which this filter was invoked.

This function sets exactly what name() will return. Subclasses should not reimplement, the default implementation should suffice.

exception bibolamazi.core.bibfilter.BibFilterError(filtername, message)

Bases: bibolamazi.core.butils.BibolamaziError

Exception a filter should raise if it encounters an error.

bibolamazi.core.bibfilter.argtypes module

class bibolamazi.core.bibfilter.argtypes.ColonCommaStrDict(*args, **kwargs)

Bases: dict

A dictionary of values, specified as a comma-separated string of pairs 'key:value'. If no value is given (no colon), then the value is None.

type_arg_input = <bibolamazi.core.bibfilter.argtypes.StrEditableArgType object>

class bibolamazi.core.bibfilter.argtypes.CommaStrList(iterable=[])

Bases: list

A list of values, specified as a comma-separated string.

type_arg_input = <bibolamazi.core.bibfilter.argtypes.StrEditableArgType object>

class bibolamazi.core.bibfilter.argtypes.EnumArgType(listofvalues)

Bases: object

option_val_repr(x)

class bibolamazi.core.bibfilter.argtypes.LogLevel(val=None)

Bases: object

One of ‘CRITICAL’, ‘ERROR’, ‘WARNING’, ‘INFO’, ‘DEBUG’, or ‘LONGDEBUG’.

levelnos = [('CRITICAL', 50), ('ERROR', 40), ('WARNING', 30), ('INFO', 20), ('DEBUG', 10), ('LONGDEBUG', 5)]

levelnos_dict = {'CRITICAL': 50, 'DEBUG': 10, 'ERROR': 40, 'INFO': 20, 'LONGDEBUG': 5, 'WARNING': 30}

levelnos_list = ['CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG', 'LONGDEBUG']

type_arg_input = <bibolamazi.core.bibfilter.argtypes.EnumArgType object>

class bibolamazi.core.bibfilter.argtypes.MultiTypeArgType(typelist, parse_value_fn)

Bases: object

option_val_repr(x)

class bibolamazi.core.bibfilter.argtypes.StrEditableArgType

Bases: object

option_val_repr(x)

bibolamazi.core.bibfilter.argtypes.enum_class(class_name, values, default_value=0, value_attr_name='value')

Define a class with the given name class_name, which can store one of several choices. The possible values are fixed. Each choice has an integer and a string representation.

class_name is the class name.

values should be a list of tuples (string_key, numeric_value) of all the expected string names and of their corresponding numeric values.

default_value should be the value that would be taken by default, e.g. by using the default constructor.

value_attr_name the name of the attribute in the class that should store the value. For example, the arxiv module defines the enum class Mode this way with the attribute mode, so that the numerical mode can be obtained with enumobject.mode.

bibolamazi.core.bibfilter.argtypes.multi_type_class(class_name, typelist, value_attr_name='value', valuetype_attr_name='valuetype', convert_functions=[(<class 'bool'>, <function getbool>)], parse_value_fn=None, doc=None)

Define a class with the given name class_name, which can store a value of one of several fixed types. This is used, for instance, in the fixes filter where for some options one can specify either “True/False” (bool type) or a list of fields (CommaStrList type).

class_name is the class name.

typelist should be a list of tuples (typeobject, description) of type objects that can be stored by this object and a corresponding very short description of what is stored with that type

default_value should be the value that would be taken by default, e.g. by using the default constructor.

value_attr_name the name of the attribute in the class that should store the value.

valuetype_attr_name the name of the attribute in the class that should store the type object that is currently stored.

Optionally, you can also specify a list of helper functions that can convert stuff into a given type: convert_functions is a list of tuples (type_object, function) that specifies this.

If parse_value_fn is not None, then it should be set to a callable that parses a value and returns a tuple (typeobject, value). It can raise ValueError.

bibolamazi.core.bibfilter.factory module

class bibolamazi.core.bibfilter.factory.DefaultFilterOptions(filtername=None, filterpath={'bibolamazi.filters': None}, finfo=None)

Bases: object

Instantiate this class as:

defopt = DefaultFilterOptions(filtername [, filterpath=...])    OR
defopt = DefaultFilterOptions(finfo=...)

filterAcceptsVarArgs()

Returns True if the filter can accept additional positional arguments.

filterAcceptsVarKwargs()

Returns True if the filter can accept additional keyword arguments.

filterDeclOptions()

This gives a list of _ArgDoc named tuples.

filterOptions()

This gives a list of _ArgDoc named tuples.

filterVarOptions()

This gives a list of _ArgDoc named tuples.

filtername()

format_filter_help()

getArgNameFromSOpt(x)

getSOptNameFromArg(x)

optionSpec(argname)

parse_optionstring(optionstring)

Parse the given option string (one raw string, which may contain quotes, escapes etc.) into arguments which can be directly provided to the filter constructor.

parse_optionstring_to_optspec(optionstring)

Parses the optionstring, and returns a description of which options where specified, which which values.

This doesn’t go as far as parse_optionstring(), which returns pretty much exactly how to call the filter constructor. This function is meant for example for the GUI, who needs to parse what the user specified, and not necessarily how to construct the filter itself.

Return a dictionary:

{
  "_args": <additional *pargs positional arguments>
  "kwargs": <keyword arguments>
}

The value of _args is either None, or a list of additional positional arguments if the filter accepts *args (and hence the option parser too). These will only be passed to *args and NOT be distributed to the declared arguments of the filter constructor.

The value of kwargs is a dictionary of all options specified by keywords, either with the --keyword=value syntax or with the syntax -sKey=Value. The corresponding value is converted to the type the filter expects, in each case whenever possible (i.e. documented by the filter).

NOTE: This function doesn’t actually validate all options to check whether the filter will accept them (e.g., options like ‘-sKey=Value’ will be blindly appended to the kwargs). See FilterInfo.validateOptionStringArgs() for that.

parser()

use_auto_case()

class bibolamazi.core.bibfilter.factory.FilterArgumentParser(filtername, **kwargs)

Bases: argparse.ArgumentParser

error(message: string)

Prints a usage message incorporating the message to stderr and exits.

If you override this in a subclass, it should not return – it should either exit or raise an exception.

exit(status=0, message=None)

exception bibolamazi.core.bibfilter.factory.FilterCreateArgumentError(errorstr, name=None)

Bases: bibolamazi.core.bibfilter.factory.FilterError

Although the filter arguments may have been successfully parsed, they may still not translate to a valid python filter call (i.e. in terms of function arguments, for example when using both positional and keyword arguments). This error is raised when the composed filter call is not valid.

fmt(name)

exception bibolamazi.core.bibfilter.factory.FilterCreateError(errorstr, name=None)

Bases: bibolamazi.core.bibfilter.factory.FilterError

There was an error instantiating the filter. This could be due to the filter constructor itself raising an exception.

fmt(name)

exception bibolamazi.core.bibfilter.factory.FilterError(errorstr, name=None)

Bases: Exception

Signifies that there was some error in creating or instanciating the filter, or that the filter has a problem. (It could be, for example, that a function defined by the filter does not behave as expected. Or, that the option string passed to the filter could not be parsed.)

This is meant to signify a problem occuring in this factory, and not in the filter. The filter classes themselves should raise bibfilter.BibFilterError in the event of an error inside the filter.

fmt(name)

setName(name)

class bibolamazi.core.bibfilter.factory.FilterInfo(name, filterpath={'bibolamazi.filters': None})

Bases: object

Information about a given filter.

Arguments:

• name: the name of the filter to get information about. It may be of the form ‘filtername’ or ‘pkgname:filtername’.
• filterpath: an collections.OrderedDict of paths where to look for filters. Keys are package names and values are the filesystem directory that needs to be added to sys.path to be able to import the given filter package as a Python package.

filtername

The filter name (without any filter package information).

fmodule

The module object that contains the filter.

fclass

The class object that implements the filter.

Note that simple-syntax filters are internally translated into a class object that fulfils the full-syntax filters API. So all filters have a valid class object that behaves as one would expect.

filterpackagename

The name of the filter package in which this filter is located. This value is always valid. The built-in filters are in the filter package bibolamazi.filters.

filterpackagedir

The filesystem path that needs to be added to sys.path in order to be able to import the filter package filterpackagename as a Python package. Note: bibolamazi assumes that sys.path is not being mischieviously manipulated by other parts of the Python code. If the filter package can be loaded from Python’s default path, this directory may not be accurate.

filterpackagespec

A string of the form ‘pkgname=file/system/path’ that combines the information of filterpackagename and filterpackagedir in a single string. This is a valid argument to the function parse_filterpackage_argstr().

This object also exposes the following attributes, which reflect what arguments were given to the constructor.

name

The filter name, as specified to the constructor.

filterpath

The filter path specified to the constructor (or the default one), that was used to resolve the information to the present filter.

The following methods are available for more specific filter information.

defaultFilterOptions()

Return a DefaultFilterOptions object that is capable of standard filter argument parsing for this filter.

This method returns None if the filter doesn’t use the default argument parsing mechanism.

formatFilterHelp()

Get the filter’s help text.

This is either the filter’s own custom help text, or the help text retrieved from the filter’s argument parser.

static initFromModuleObject(filtername, module_obj, fpkgname, fpkgdir)

Initializes a FilterInfo object from an already-imported module object module_obj in a given filter package with name fpkgname residing in dir fpkgdir. The module name without the package information must be given in filtername.

makeFilter(optionstring)

Instantiate the filter with the given option string.

Returns the new filter instance.

Raises py:exc:FilterCreateError if any exception was raised during the filter instantiation.

parseOptionStringArgs(optionstring)

validateOptionStringArgs(pargs, kwargs)

Validate the arguments as OK to pass to constructor, i.e. that all argument names are correct.

We use inspect.getcallargs() to inspect the filter class constructor’s signature.

Raises FilterCreateArgumentError if the validation fails.

exception bibolamazi.core.bibfilter.factory.FilterOptionsParseError(errorstr, name=None)

Bases: bibolamazi.core.bibfilter.factory.FilterError

Raised when there was an error parsing the option string provided by the user.

fmt(name)

exception bibolamazi.core.bibfilter.factory.FilterOptionsParseErrorHintSInstead(errorstr, name=None)

Bases: bibolamazi.core.bibfilter.factory.FilterOptionsParseError

As FilterOptionsParseError, but hinting that maybe -sOption=Value was meant instead of -dOption=Value.

fmt(name)

class bibolamazi.core.bibfilter.factory.FilterPackageSpec(a, b=None)

Bases: object

Stores a filter package specification, and provides a convenient API to download remote packages.

Constructor: FilterPackageSpec(argstr) or FilterPackageSpec(path) or FilterPackageSpec(url) or FilterPackageSpec(fpname, fpdir)

Properties:

• is_url – True if the filter package was specified directly as a path to the python package, or as a URL. False if individual filter names and python-path were specified.
• url – if is_url==True, then this is the path or URL that was specified.
• fpname – the name of the filter, may be None if is_url==True until materialize() is called.
• fpdir – the directory which needs to be added to sys.path to load the filter package, may be None if is_url==True until materialize() is called.

New in version 4.2: Added FilterPackageSpec class.

materialize()

Ensures that the filter package is present on the local filesystem, if it is a remote filesystem, and returns specific information on how to load it.

This method may be called regardless of whether the spec was originally an URL or not, i.e., for either value of is_url the return value of this is correct.

Returns a tuple (fpname, fpdir) of a filter package name and filesystem directory which needs to be added to the sys.path in order to load the former.

repr()

exception bibolamazi.core.bibfilter.factory.ModuleNotAValidFilter(fname, errorstr=None)

Bases: bibolamazi.core.bibfilter.factory.NoSuchFilter

Signifies that a given module does not expose a valid bibolamazi filter. See also FilterInfo.

exception bibolamazi.core.bibfilter.factory.NoSuchFilter(fname, errorstr=None)

Bases: Exception

Signifies that the requested filter was not found. See also FilterInfo.

exception bibolamazi.core.bibfilter.factory.NoSuchFilterPackage(fpname, errorstr='No such filter package', fpdir=None)

Bases: Exception

Signifies that the requested filter package was not found. See also FilterInfo.

class bibolamazi.core.bibfilter.factory.PrependOrderedDict(*args, **kwargs)

Bases: collections.OrderedDict

An ordered dict that stores the items in the order where the first item is the one that was added/modified last.

item_at(idx)

set_at(idx, key, value)

set_items(items)

bibolamazi.core.bibfilter.factory.detect_filter_package_listings(force_redetect=False, filterpath={'bibolamazi.filters': None})

bibolamazi.core.bibfilter.factory.detect_filters(force_redetect=False, filterpath={'bibolamazi.filters': None})

bibolamazi.core.bibfilter.factory.format_filter_help(filtname=None, filterpath={'bibolamazi.filters': None})

Format help text for the given filter. This is a shortcut to the corresponding method in FilterInfo.

bibolamazi.core.bibfilter.factory.inspect_getargspec(f)

bibolamazi.core.bibfilter.factory.load_precompiled_filters(filterpackage, precompiled_modules)

filterpackage: name of the filter package under which to scope the given precompiled

filter modules.

precompiled_modules: a dictionary of ‘filter_name’: filter_module of precompiled

filter modules, along with their name.

bibolamazi.core.bibfilter.factory.make_filter(name, options, filterpath={'bibolamazi.filters': None})

Main filter instance factory function: Create the filter instance.

This is a simple wrapper for FilterInfo(…).makeFilter(…).

bibolamazi.core.bibfilter.factory.package_provider_manager = None

The package provider manager. If we a filterpackage is specified with a URL, then this is the manager we use to retreive the remote filter package.

The main module is responsible for instantiating a PackageProviderManager instance and storing it here.

bibolamazi.core.bibfilter.factory.parseArgdoc(doc)

Parses argument documentation from a docstring. Extracts lists of argument documentation in a relatively crude way. Expects arguments to be documented in lines of the form:

* argument_name (type) :  Here comes the documentation of the argument. It
  may span several lines, that is ok.
*arg2: as you can see, whitespace is optional and ignored; the type
       name is also not necessary.
- arg3: also, argument listings may begin with a dash instead of an asterisk

The argument list is expected to be a the end of the docstring. I.e., anything that follows the argument list will be included in the doc of the last argument!

If there is a line with the single word ‘Arguments’ (with possible punctuation), e.g.:

Arguments:

Then argdocs are processed only after that line.

Returns a tuple (argdoclist, fndoc). The argdoclist is a list of objects (actually, named tuples), with the fields ‘argname’, ‘argtypename’, and ‘doc’. If an argtypename is absent, it is set to an empty value. The fndoc is a string corresponding to the rest of the docstring before the argument documentation.

bibolamazi.core.bibfilter.factory.parse_filterpackage_argstr(argstr)

Parse filter package specification given as “path/to/the/package” or as a URL. The format “filterpackage=path/to/the/package” or “filterpackage=” is also accepted for local filesystem paths, but not for URLs.

Returns a tuple (fpname, fpdir) of a filter package name and filesystem directory which needs to be added to the sys.path in order to load the former.

Note that if a remote URL is provided, then it is downloaded and stored in cache.

Deprecated since version 4.2: Use the FilterPackageSpec class instead.

bibolamazi.core.bibfilter.factory.reset_filters_cache()

bibolamazi.core.bibfilter.factory.validate_filter_package(fpname, fpdir, raise_exception=True)

Make sure given filter package at given directory is valid.

Utility to warn the user of invalid –filterpackage option