Private API
Abstract Data Transformer
DataToolkitBase.AbstractDataTransformer
— TypeThe supertype for methods producing or consuming data.
╭────loader─────╮
╵ ▼
Storage ◀────▶ Data Information
▲ ╷
╰────writer─────╯
There are three subtypes:
DataStorage
DataLoader
DataWrite
Each subtype takes a Symbol
type parameter designating the driver which should be used to perform the data operation. In addition, each subtype has the following fields:
dataset::DataSet
, the data set the method operates ontype::Vector{<:QualifiedType}
, the Julia types the method supportspriority::Int
, the priority with which this method should be used, compared to alternatives. Lower values have higher priority.parameters::SmallDict{String, Any}
, any parameters applied to the method.
Advice Amalgamation
DataToolkitBase.AdviceAmalgamation
— TypeA collection of Advices
sourced from available Plugins.
Like individual Advices
, a AdviceAmalgamation
can be called as a function. However, it also supports the following convenience syntax:
(::AdviceAmalgamation)(f::Function, args...; kargs...) # -> result
Constructors
AdviceAmalgamation(adviseall::Function, advisors::Vector{Advice},
plugins_wanted::Vector{String}, plugins_used::Vector{String})
AdviceAmalgamation(plugins::Vector{String})
AdviceAmalgamation(collection::DataCollection)
Qualified Types
DataToolkitBase.QualifiedType
— TypeA representation of a Julia type that does not need the type to be defined in the Julia session, and can be stored as a string. This is done by storing the type name and the module it belongs to as Symbols.
While QualifiedType
is currently quite capable, it is not currently able to express the full gamut of Julia types. In future this will be improved, but it will likely always be restricted to a certain subset.
Subtyping
While the subtype operator cannot work on QualifiedTypes (<:
is a built-in), when the Julia types are defined the subset operator ⊆
can be used instead. This works by simply convert
ing the QualifiedTypes to the corresponding Type and then applying the subtype operator.
julia> QualifiedTypes(:Base, :Vector) ⊆ QualifiedTypes(:Core, :Array)
true
julia> Matrix ⊆ QualifiedTypes(:Core, :Array)
true
julia> QualifiedTypes(:Base, :Vector) ⊆ AbstractVector
true
julia> QualifiedTypes(:Base, :Foobar) ⊆ AbstractVector
false
Constructors
QualifiedType(parentmodule::Symbol, typename::Symbol)
QualifiedType(t::Type)
Parsing
A QualifiedType can be expressed as a string as "$parentmodule.$typename"
. This can be easily parse
d as a QualifiedType, e.g. parse(QualifiedType, "Core.IO")
.
Global variables
DataToolkitBase.STACK
— ConstantThe set of data collections currently available.
DataToolkitBase.PLUGINS
— ConstantThe set of plugins currently available.
DataToolkitBase.EXTRA_PACKAGES
— ConstantThe set of packages loaded by each module via @addpkg
, for import with @import
.
More specifically, when a module M invokes @addpkg pkg id
then EXTRA_PACKAGES[M][pkg] = id
is set, and then this information is used with @import
to obtain the package from the root module.
DataToolkitBase.DATA_CONFIG_RESERVED_ATTRIBUTES
— ConstantThe data specification TOML format constructs a DataCollection, which itself contains DataSets, comprised of metadata and AbstractDataTransformers.
DataCollection
├─ DataSet
│ ├─ AbstractDataTransformer
│ └─ AbstractDataTransformer
├─ DataSet
⋮
Within each scope, there are certain reserved attributes. They are listed in this Dict under the following keys:
:collection
forDataCollection
:dataset
forDataSet
:transformer
forAbstractDataTransformer