autoextract_poet.adapters.AutoExtractAdapter

class AutoExtractAdapter(item: Any)

Bases: itemadapter.adapter.AttrsAdapter

ItemAdapter for AutoExtract poet items that deals transparently with both the attr.s defined fields and the rest of unknown fields received at item initialization, offering an unified view over all the item fields.

The utility is twofold. Firstly, it serves to ensure the pass-through of new fields from the API even if autoextract-poet item definitions have not been yet updated. In other words, it can be used to create spiders that preserve all the data coming from the API even if they don’t have updated item definitions.

Secondly, it offers a common interface to access and modify both kind of fields (known and unknown).

Remember that this adapter should be enabled by invoking:

ItemAdapter.ADAPTER_CLASSES.appendleft(AutoExtractAdapter)

__init__(item: Any) → None

Methods

__init__(item)
clear()
field_names()	Return a dynamic view of the item's field names.
get(k[,d])
get_field_meta(field_name)	Return metadata for the given field name, if available.
get_field_meta_from_class(item_class, field_name)
is_item(item)	Return True if the adapter can handle the given item, False otherwise.
is_item_class(item_class)	Return True if the adapter can handle the given item class, False otherwise.
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised.
popitem()	as a 2-tuple; but raise KeyError if D is empty.
setdefault(k[,d])
update([E, ]**F)	If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
values()

Attributes

field_names() → KeysView

Return a dynamic view of the item’s field names.

get_field_meta(field_name: str) → mappingproxy

Return metadata for the given field name, if available.

classmethod is_item(item: Any) → bool

Return True if the adapter can handle the given item, False otherwise.