Changelog
See https://github.com/Fraunhofer-IESE/badgers/releases
Unreleased
Fixed
- MNAR missingness direction —
_mnar_weightsnow normalizes forward ((X - X.min) / range), matching the documented semantics: HIGH values of a column are the more likely to be missing (previously the normalization was reversed, making low values the more likely to be missing). Constant columns now get uniform missingness weights (scale 1.0 fallback) in both the MAR and MNAR mechanisms instead of raising aFloatingPointErroron the 0/0 division. ExactCountSelectoratrate=1.0— asking for exactly all rows with weighted sampling no longer raises (ValueError: Fewer non-zero entries in p than sizefromnumpy.random.Generator.choice); all rows are returned directly.
Added
UndersamplingGenerator(inbadgers.generators.tabular_data.imbalance) — a row-removing imbalance generator:generate(X, selector=None)removes the rows flagged by a deficitSelector(True/1.0= row removed; fractional specs are removed stochastically; multiple columns = union;selector=Noneis a no-op).get_imbalance_labels()returns a boolean array aligned with the input rows (True= row removed).OversamplingGenerator(inbadgers.generators.tabular_data.imbalance) — a row-duplicating imbalance generator:generate(X, selector=None)duplicates rows according to per-row integer multiplicities (ConstantMultiplicity(k),MaskedMultiplicity(mask, k), or a bare int /(n,)int array; all values must be>= 1). The output is a superset of the input — no row is ever dropped and the original order is preserved.get_imbalance_labels()returns a boolean array aligned with the input rows (True= row duplicated, i.e. multiplicity > 1).ConstantMultiplicity(k)— aMultiplicitiesSelectorthat assigns every row multiplicityk.MaskedMultiplicity(mask, k)— aMultiplicitiesSelectorthat assigns multiplicitykto the rows wheremaskisTrue(a(n,)bool array or a callable onX) and multiplicity 1 to the rest.OutlierSamplingGenerator(inbadgers.generators.tabular_data.outliers.sampling) — the generic non-causal outlier generator:generate(X, selector, *, out_of_distribution_sampler="zscore")appends rows that are copies of the selector's seed rows with the selector-selected cells replaced by out-of-distribution values (all other cells keep the seed row's values). Theselectoris REQUIRED — it owns the outlier count and the perturbed targets (e.g.CountSelector([0, 1], count=20, rng=...)).get_outliers_labels()returns a boolean array aligned with the OUTPUT rows (True= appended outlier row).KernelDensitySampler(inbadgers.core.sampling, registry name"low_density") — a KDE-based out-of-distribution sampler preserved from the removedLowDensitySamplingGenerator. Rewritten as a grid-CDF construction (no rejection): it scores a bounded regular grid of at most ~grid_budgetcells (hard cap of 65536 cells;NotImplementedErrorif fewer than 2 cells per dimension is feasible) with an sklearnKernelDensity(Scott bandwidth,ball_tree), then draws exactlyn_samplespoints from a categorical over the grid cells at or below thethreshold_low_densitypercentile of the data log-densities, each point independently uniform inside its cell (fallback: the least-dense cells). The newgrid_budgetparameter replaces the earliermax_samples; the count is always exactlyn_samplesand deterministic for a fixedrngseed.TreeCellSampler(inbadgers.core.sampling, registry name"tree") — an adaptive-histogram out-of-distribution sampler (no KDE): recursive median splits partition the standardized columns into kd cells holding at mostleaf_sizepoints (fine resolution in dense regions, coarse in sparse ones); every cell is extended axis-aligned to the midplanes with the neighbouring points (an axis-aligned Voronoi approximation), and exactlyn_samplespoints are drawn from a categorical over the cells whose extended-territory density (count over extended volume) is at mostthreshold_low_densitytimes the global mean density (fallback: the least-dense 10 cells). Each cell is extended across the full gap to the nearest data point on both sides of every axis (the outermost cells reach the data-box boundary), so leaves tile the padded box instead of only its interior; a bounded coverage grid finds every box region not covered by any extended leaf and adds it as an always-qualifying count-0 cell, so the empty space around and between the clusters is reachable too. Weights are proportional to the extended cell volumes (uniform over the qualifying space). Deterministic for a fixedrngseed.
Changed
- Architecture documentation restructured. The monolithic
docs/architecture.mdis now a short index overdocs/architecture/(system context, building blocks, runtime scenarios, quality goals, causal design) plus a decision log indocs/architecture/adr/. The layering is additionally enforced byimport-lintercontracts inpyproject.toml(lint-imports), which now runs in CI and intox. - The imbalance API is split into under- and oversampling (breaking).
Imbalance is no longer a single score-driven resampler. Deficit-based row
removal lives in
UndersamplingGenerator(a deficitSelectordecides which rows are dropped); row duplication lives inOversamplingGenerator(a per-row multiplicity selector decides how many copies of each row survive). Oversampling input validation rejects fractional values in(0, 1)("looks like a deficit Selector"), zeros ("row dropped"), negative values ("must be >= 1"), and non-integer multiplicities ("not supported"). - The non-causal outlier generators collapsed into a single generic
OutlierSamplingGenerator(breaking). It mirrors the causalCausalOutlierPropagationGenerator's contract: a REQUIREDselector(owns the outlier count and the perturbed targets) plus a pluggableout_of_distribution_sampler(anOutOfDistributionSamplerinstance frombadgers.core.sampling, or its registry name as a string, default"zscore"). The former strategy-specific generators are now samplers:ZScoreSampler("zscore"),HypersphereSampler("hypersphere"),UniformOutOfDistributionSampler("uniform"),IndependentHistogramSampler("independent_histogram"),JointHistogramSampler("histogram"), andKernelDensitySampler("low_density").
Removed
RandomSamplingClassesGenerator— replaced byUndersamplingGenerator(with aClassSelector) andOversamplingGenerator(with aMaskedMultiplicity).RandomSamplingTargetsGenerator— replaced byUndersamplingGenerator(score-driven deficit selectors such asProportionalSelector) andOversamplingGenerator.CausalImbalanceGenerator— removed frombadgers.generators.tabular_data.causal; causal no longer ships an imbalance generator (row removal/duplication is an observation-process operation inbadgers.generators.tabular_data.imbalance).HyperCubeSampling— replaced byOutlierSamplingGeneratorwithUniformOutOfDistributionSampler(expansion)(registry name"uniform").ZScoreSamplingGenerator— replaced byOutlierSamplingGeneratorwithZScoreSampler(scale)(registry name"zscore").HypersphereSamplingGenerator— replaced byOutlierSamplingGeneratorwithHypersphereSampler(scale)(registry name"hypersphere").IndependentHistogramsGenerator— replaced byOutlierSamplingGeneratorwithIndependentHistogramSampler(bins)(registry name"independent_histogram").HistogramSamplingGenerator— replaced byOutlierSamplingGeneratorwithJointHistogramSampler(bins, threshold_low_density)(registry name"histogram").LowDensitySamplingGenerator— replaced byOutlierSamplingGeneratorwithKernelDensitySampler(threshold_low_density, grid_budget)(registry name"low_density").
No backward compatibility is provided for any of these classes.
See the new Imbalance tutorial
for the replacement patterns, and the
Outliers tutorial for the
OutlierSamplingGenerator + sampler patterns.
dev_0_0_15 (unreleased)
Breaking changes
- Selectors work on column positions only (breaking).
Every selector preset now takes integer column positions at construction
and returns a
dict[int, spec]keyed by position:AlwaysSelector,BernoulliSelector,ExactCountSelector,TopQuantileSelector,ClassSelector,ProportionalSelector. Variable names and the selector-sidecolumn_mappingare gone; read columns (class_column,column,driver) are independent int positions. Value-free presets return selection decisions (bool masks;AlwaysSelector's constant scalar1.0realizes to all-true); value-driven presets return[0, 1]selection specs (float scalar or(n,)array) that generators realize with the single rulerng.random(n) < spec. Direct generators no longer takecolumn_mapping—AdditiveNoiseGenerator.generate(X, distribution, noise_std=1.0, *, selector=None). Causal generators' call-time targets are owned by the selector itself: itscolumns(column positions) are the intervention targets, soperturbation_nodesis gone from their signatures. - Wave 2: the
get_selected_fields()field unification and selector-owned causal targets. The three realization helpers collapse into one —badgers.core.selectors.get_selected_fields(X, selector, rng)returns the narrow bool field(n_rows, n_targets)(columni↔ the selector'si-th target); row-level union isfield.any(axis=1)(fired_maskandfired_rowsare deleted). All causal generators now require a selector at call time —perturbation_nodesis removed from their signatures (CausalImbalanceGenerator's legacyperturbation_nodes+quantile_rangefallback is the one exception). Drift'sdrift_stdis replaced byshift_amount(deterministic signed shift, scalar or dict keyed by column position), andCausalNoiseGenerator.noise_stdis scalar-or-dict-by-position as well.CausalOutlierPropagationGeneratornow requires a selector (per-cell pinning: the selection pins a single cell per selected row, pure-bootstrap rows are labeledFalse, and rows sharing a selection pattern are perturbed as one group). - Wave 3: the outliers generators consume selectors; outlier rows are seeded from real rows.
All six tabular outliers generators take a selector and realize it through the
shared selection field.
CausalOutlierPropagationGenerator.generate(X, selector, *, out_of_distribution_sampler=None): the selector is required and realizes onX— the selected rows are appended as COPIES (an unselected cell keeps the seed row's value; descendants of selected targets recompute through the DGP; labels are allTrue);n_outliersandwithin_distribution_samplerare removed (CountSelector([x], count=10)replacesn_outliers=10). The direct generators (HyperCubeSampling,ZScoreSamplingGenerator,HypersphereSamplingGenerator) make selectors the only selection mechanism:n_outliersis only the DEFAULT count (selector=Nonebuilds an internalCountSelector, default 10), passing both raises, and appended rows are copies of selected rows with selected cells overwritten by the same samplers as before (same-seed values shift: the selector machinery consumes draws first). The low-density trio (IndependentHistogramsGenerator,HistogramSamplingGenerator,LowDensitySamplingGenerator) uses selector KEYS as the synthesized columns (mask discarded) and fills unselected columns with per-cell bootstrap copies.UniformInstanceAttributeSamplingacceptsselectorfor API uniformity but ignores it (warning;n_outliersstays its count). Everywhere in the family a requested count is capped at the dataset size (count > len(X)raises), 0 is a no-op and a negative count raises. - The
(X, y)contract is removed; generators and selectors takeXonly (breaking). Every generator's signature isgenerate(X, **params)and it returns exactly one generated objectXt— the oldyparameter and the two-tuple return are gone (a hard break, no deprecation shims). Every selector is called asselector(X)and returns the per-columndict[int, spec]mapping (positions keyed; bool masks from value-free presets,[0, 1]selection specs from value-driven presets).Pipeline.generate(X, params=None)threads each single-output step into the next. Thepreprocess_inputsdecorators wrapgenerate(self, X, **kwargs). Benchmark scenario factories return oneX. Target-like data lives insideX: dedicated columns for tabular and time-series data (class columns read via the selector'sclass_columnint position), node/edge attributes for graphs, and modality-native values for text. Deficit labels remain available through the generators' own getters (get_drift_labels(),get_missing_labels(),get_imbalance_labels(),get_noise_labels(),get_outliers_labels()). Supported Python versions are unchanged (3.11–3.14). - Causal generators: labels are internal generator state (breaking).
The causal defect generators
(
CausalDriftGenerator,CausalMissingnessGenerator,CausalImbalanceGenerator,CausalNoiseGenerator,CausalAdditiveNoiseGenerator) no longer return deficit labels:generate(X, **params)returns exactly one generated objectXtand each generator exposes its own boolean label getter aligned with the rows of the last generated output — (get_drift_labels(),get_missing_labels(),get_imbalance_labels(),get_noise_labels()(both noise generators),get_outliers_labels(). Calling a getter beforegenerate()raisesRuntimeError. The base class owns the storage slot (deficit_labels_); each generator owns its getter. CausalGenerator._validatereplaced by two raise-only helpers._validate(X, perturbation_nodes)(which mixed validation with graph-derivative computation and returned a 5-tuple) is split into_validate_structure(X)(graph/DAG, column count, column_mapping coverage and indices) and_validate_perturbation_nodes(nodes)(non-empty list of known nodes). Neither returns anything — the mapping isself.column_mapping. The deadordermember and the graph algebra (descendants/exogenous/perturb_cols) moved into the only generator that used them (CausalOutlierPropagationGenerator).PyMCDGPand the oldbadgers.core.dgpmodule are removed. The causal defect generators now fit a per-node data generation process behind the newDGPprotocol (badgers.core.dgpis a package). Backends are injected as objects (dgp=.../use_dgp); PyMC-specific parameters (n_samples_mcmc,fit_dgp) no longer appear infit(...).CausalGenerator.fitremoved AND thegraph=/column_mapping=generator-side structure is gone (breaking). The DGP is the ONLY structure a causal generator accepts: the constructor isCausalGenerator(random_generator=..., dgp=None)and the only structure param anywhere on the generator isdgp=(oruse_dgp(dgp)). The DGP carriesgraphandcolumn_mappingitself — build one with e.g.dgp = GcmSCMDGP(scm).fit(X)(seedocs/tutorials/dev/Using-DGPs.ipynb). Unfitted DGPs are fitted lazily on firstgenerate(X)(DGP.is_fittedis respected).fitsignature change for the causal defect family:fit(X, graph, column_mapping, dgp=None)—n_samples_mcmcandfit_dgpare gone.AdditiveNoiseGenerator(new, non-causal): injects noise drawn from any frozen scipy distribution object (distribution=....rvs(size, random_state=rng)) instead of hard-coded Gaussian white noise (e.g.scipy.stats.laplace(scale=1),scipy.stats.t(df=3)).noise_stdis a global magnitude multiplier (1.0 = raw draw); what you pass is what is added — no rescaling. Optionalperturbation_nodes` restricts the noise to a column subset.CausalAdditiveNoiseGenerator(new, causal): same swappable distribution at the perturbation nodes, with descendant recomputation via the DGP (dgp.intervene— Pearl surgery). Row selection is delegated to aselector(default: every row is selected); strictly in-place (non_copies).GaussianNoiseGeneratorremoved (breaking). It is a strict special case of the newAdditiveNoiseGenerator: replaceGaussianNoiseGenerator().generate(X, noise_std=0.5)withAdditiveNoiseGenerator().generate(X, distribution=scipy.stats.norm(scale=0.5)). (Note the semantic shift: the old class standardized first, so itsnoise_stdwas column-relative; the new API adds the frozen distribution's raw draws — pass the per-column scale yourself.)GaussianNoiseClassesGenerator(per-class noise) remains; the time-seriesLocal/GlobalGaussianNoiseGeneratorare unrelated and unchanged.CausalEdgePerturbationGeneratoris removed for this release (deferred to future work). The relationship-level (edge weight) defect — scaling one edge coefficient and recomputing the child marginal-preserving — was developed behind the family API but is not part of the 0.0.15 surface: the generator, its tests and the "Perturbing Edge Strength" tutorial are removed. The underlyingedits.scale_edgeDGP edit remains inbadgers.core.dgp.edits(and is tested), so re-introducing the generator is a small follow-up.CausalImbalanceGeneratorno longer requires a DGP (breaking): the generator is a pure row-subset (quantile-range truncation); the kept rows' parents are unchanged, so no structural equation is re-evaluated. The deadmechanism/mechanism_cause/mechanism_rateparameters, the on-demand OLS fit, and the no-op propagate pass are removed — the signature is nowgenerate(X, perturbation_nodes=None, quantile_range=(0.5, 1.0)). This is the first generator in the causal family whosegenerateneeds neither a DGP nor any random numbers (selection-side only).CausalMissingnessGeneratoris chooser-only (breaking): missingness is an observation-layer defect — it only hides cells (writes NaN) and does NOT cascade to descendants. Themissingness_rulemechanism parameter, the on-demand DGP fit, and theany/allNaN-propagation passes are removed; the selection comes from aselectorobject (see the selector bullet below). Signature:generate(X, perturbation_nodes=None, selector=None).- String
mechanismflags replaced byselectorobjects (breaking). The row-selection parameters of the causal defect generators (CausalDriftGenerator,CausalMissingnessGenerator,CausalNoiseGenerator,CausalAdditiveNoiseGenerator) are now a singleselector=parameter holding a callable frombadgers.core.selectors;Nonedefaults to "every row is selected" (AlwaysSelectorsemantics). Themechanism/mechanism_cause/mechanism_ratestrings and the_fire_masksstring dispatcher are removed. Migration (pos...= the perturbation nodes' column positions from the DGP'scolumn_mapping): mechanism="always"→ omitselector(orAlwaysSelector([pos...])).mechanism="random", mechanism_rate=r→selector=BernoulliSelector([pos...], rate=r, rng=gen.random_generator).mechanism="self", mechanism_rate=r→selector=TopQuantileSelector([pos...], column=pos, quantile=1-r)(value-driven presets read the oneposand broadcast it).mechanism="observed", mechanism_cause=c, mechanism_rate=r→selector=TopQuantileSelector([pos...], column=pos, quantile=1-r, driver=pos_c)withpos_cthe cause column's position from the DGP'scolumn_mapping. Intensity lives on the selector object:rateandquantileare now CONSTRUCTOR arguments ofBernoulliSelector/ExactCountSelector/TopQuantileSelector(previouslyratewas a__call__kwarg andquantilea call kwarg) — so an intensity sweep means constructing a new selector.AlwaysSelectorwas added (the all-rows preset).RandomSamplingClassesGeneratortakes aselector(breaking). Theproportion_classescall kwarg is removed; pass insteadselector=ClassSelector([pos], class_column=pos, proportions={...}, rng=...)frombadgers.core.selectors, withposthe class column's int position inXandproportions={class_value: proportion}. Class columns are columns ofX: selection reads onlyX(the class column is the selector'sclass_columnint position). The selector's selection is the standard contract (True = deficit); the generator's deficit is DROPPING the selected rows (subsampling), so the output is smaller than the input. Classes absent fromproportionsare kept as-is; proportions are shares of the total row count, clipped to the rows available in each class. The class variable must be a column ofX, given as the selector'sclass_columnint position.selector=Noneis a no-op. Migration:generate(X, proportion_classes={0: 0.5, ...})→generate(np.column_stack([X, y]), selector=ClassSelector([pos], class_column=pos, proportions={0: 0.5, ...}, rng=rng))withposthe position of the stacked target column. Note the new semantics sample WITHOUT replacement (the old implementation duplicated rows when a proportion exceeded the class size).- Outlier labels are internal generator state, no longer returned by
generate(breaking). All outlier generators (HyperCubeSampling,ZScoreSamplingGenerator,HypersphereSamplingGenerator,UniformInstanceAttributeSampling,IndependentHistogramsGenerator,HistogramSamplingGenerator,LowDensitySamplingGenerator,CausalOutlierPropagationGenerator, andStructuredOutlierGenerator) now exposeget_outliers_labels(): a boolean vector aligned with the rows of the LAST generated output,Truewhere the row is an outlier,Falsewhere it is an original row.generate(X, **params)returns exactly one generated objectXt(the previous"original"/"outliers"string arrays are gone). Callingget_outliers_labels()beforegenerate()raisesRuntimeError. Migration: the old two-tuple unpacking fromgen.generate(X)followed by string-comparing the second element →Xt = gen.generate(X)followed bygen.get_outliers_labels(). - Selector scores may now be fractional (additive contract extension).
A selector's per-row value may be a float in
[0, 1]— the PROBABILITY that the deficit is applied — where boolean masks remain the degenerate (deterministic) case. Realization belongs to the consuming generator (a modifying consumer is realized Bernoulli-wise, a resampling consumer uses the scores as selection weights); a preset's own rng only draws its spec (see the index-contract bullet above). TheProportionalSelector([pos], column=pos)preset returnsnormalize_proba(X[:, pos])(the deficit probability proportional to a column's value). RandomSamplingTargetsGeneratortakes aselector(breaking). Thesampling_proba_func(y)kwarg is removed; the generator now resamples rows weighted by1 - deficit_probabilityfrom a selector (noisy-OR combined over its variables), i.e. rows with a high deficit probability are dropped/under-sampled, rows with a low deficit probability are kept or replicated. Signature:generate(X, *, selector=None, replace=True, size=None);replace=True(default) may duplicate rows (importance resampling),replace=Falsegives a weighted thinning,sizecontrols the output row count (default: input row count). Selection reads onlyX(per-column specs keyed by the columns' int positions). Migration: the old default (sample ∝ target values) becomesProportionalSelector(idx, column=idx)over whicheverXcolumn holds the signal (e.g. the stacked target column).RandomSamplingFeaturesGeneratorremoved (breaking). It was the free-probability twin of the resampler; its value-weighted resample is nowRandomSamplingTargetsGenerator(selector=ProportionalSelector( idx, column=idx)), and its per-class behavior is the same generator with a boolean selector (ClassSelectorover the class column).RandomShiftClassesGeneratortakes aselector(breaking). The class variable is read from a column ofX(the selector'sclass_columnint position). Signature:generate(X, *, selector=None, shift_std=0.1). The selector is REQUIRED (selector=Noneraises) — a per-class drift has no class-free default; selection masks are OR-ed.shift_stdis now a float (one independent draw magnitude per class value) or a dict{class_value: std}; entries are SIGNED (|std|= draw scale, sign = direction). The per-dimension 2D-array form is removed (per-class shifts are isotropic — scale the data first for anisotropic drift). Class values absent from a dict are skipped. Migration:generate(X, shift_std=np.array([2., 2., 2., 2.]))→generate(np.column_stack([X, y]), selector=ClassSelector([pos], class_column=pos, proportions={c: 1.0 for c in classes}, rng=rng), shift_std=2.0)withposthe position of the stacked target column.- Drift generators: signed per-column
shift_stdand class-column protection (breaking,RandomShiftGenerator/RandomShiftClassesGenerator).shift_stdentries are now signed: the magnitude (|shift_std|) is the scale of the normal draw and the sign is the shift direction — a negative entry drifts the column downward. ForRandomShiftClassesGeneratorthe class column is identification metadata and is NEVER shifted by default (shifting it would move rows away from their class values); new parametersvariables(columns to shift) andclass_column(explicit class column position) allow shifting an explicit subset and/or a different class column. A dictshift_std={class_value: std}now requires a resolvable class column (raises otherwise). - Deterministic drift generators (new):
FixedShiftGeneratorandFixedShiftClassesGenerator— the non-random counterparts of the random shift generators. Each column is translated by exactlyshift_std(in standard-deviation units, data standardized first; sign = direction); no random number is drawn, so repeated calls with the same input return the same output. Both are exported frombadgers.generators.tabular_data.drift. GaussianNoiseClassesGeneratortakes aselector(breaking). Same contract as above: classes are a column ofX, given as the selector'sclass_columnint position (selector=Noneraises; a per-class lookup requires a selector exposingclass_columnto resolve the class column). The class column itself is never noised — the old implementation rebuilt the output's last column as class labels, silently corrupting it (bug fixed). Signature:generate(X, *, selector=None, noise_std_per_class=dict()); selected rows whose class value is absent from the dict receive no noise.do()andScaledEdgeDGPare removed from the public API. They are replaced by the intervention/edition surface on the DGP protocol itself (see Added): a single data-levelDGP.interveneand the structure edits as free functions in the newbadgers.core.dgp.editsmodule (see below).ordered_descendantsmoved tobadgers.core.causal_graph(still re-exported frombadgers.core.dgp). Consequence for the causal defect generators: descendants of an intervention are recomputed with fresh noise (Pearl surgery) instead of propagating the delta with each node's exogenous noise preserved — the produced deficits are slightly "wider" (noise re-rolled downstream).- DGP structure-edit methods removed (breaking):
scale_edge/remove_edge/exogenize/set_form(and theadd_edge/reverse_edgestubs) are no longer methods on the DGP. The DGP is an immutable, fitted model with ONLY evaluation verbs (fit/response/draw_noise/sample/intervene); all structure modifications live as FREE FUNCTIONS in the newbadgers.core.dgp.editsmodule (from badgers.core.dgp import edits):edits.scale_edge(dgp, child, parent, k),edits.remove_edge(dgp, child, parent),edits.add_edge_contribution(dgp, child, parent, contribution=... or coefficient=..., draw_noise=...)(the A5 cheap edit, ANM-scoped; cycle and existing-edge checks),edits.exogenize(dgp, node)(INTERVENTION-ONLY semantics — see below),edits.set_form(dgp, node, response_fn, draw_noise=None)(Q2: an optional noise-law closure). Edits return read-only views and never mutate the base DGP; they compose by delegation. - Every generator requires an explicit
random_generator(breaking). No generator constructor has a random-generator default anymore —AdditiveNoiseGenerator()now fails withTypeError; pass one:AdditiveNoiseGenerator(random_generator=numpy.random.default_rng(0)). The olddefault_rng(seed=0)default was evaluated ONCE at import time, so all instances constructed without an explicit rng shared ONE Generator object (two generators created without an rng advanced each other's streams). Each generator now owns exactly the stream its user gave it. The two deterministic drift generators (FixedShiftGenerator,FixedShiftClassesGenerator) take an rng too (realizing the selector's specs consumes one; the shift itself still draws nothing) — repeated calls with the same input return the same output. - Causal generators require the selector as a positional parameter
(breaking).
CausalNoiseGenerator.generate(X, selector, ...)andCausalAdditiveNoiseGenerator.generate(X, selector, ...)no longer defaultselectortoNone: omitting it is aTypeError, passingNoneexplicitly raisesValueError(the selector owns the intervention targets). Same contract as the drift, missingness and outlier causal generators (which already declared it positionally); the one exception isCausalImbalanceGenerator's legacyselector=Nonequantile-range path.
Added
- New OOD samplers
IndependentHistogramSamplerandJointHistogramSampler(badgers.core.sampling, factory names"independent_histogram"and"histogram") — per-column and joint low-density histogram sampling, extracted from the outlier generatorsIndependentHistogramsGeneratorandHistogramSamplingGenerator, which now delegate to them with bit-for-bit identical seeded synthesis and plug into the causal outlier generator as-is (55502c4,caeb0e7,9054f30). - Exact-count selector presets:
CountSelector(columns, count, rng)— exactlycountdistinct rows with one shared mask under every target (count > len(X)raises);ClassCountSelector(columns, class_column, counts, rng)— exact row counts per class value (unknown class values and per-class overflow raise). -
New within-distribution sampler
InstanceSampler— per-cell bootstrap (each cell copies the value of a random real row); extracted fromUniformInstanceAttributeSampling, which now delegates to it (bit-identical values). -
New DGP backend
GcmSCMDGP(dowhy extra): plug an externally builtdowhy.gcm.StructuralCausalModelinto badgers — mechanism fitting is delegated togcm.fit, sampling and interventions run on the generic protocol with caller-provided rngs. SupportsAdditiveNoiseModel(non-root),DiscreteAdditiveNoiseModel(non-root count nodes — discrete predictions are rounded so the composition stays integral),EmpiricalDistribution(root) andClassifierFCM(non-root discrete nodes with numeric-encoded classes). Cascade composition is delegated to the mechanism's ownevaluate(parents, noise)via the DGP'sstructuralhook — badgers draws the noise, dowhy owns the math. - New DGP backend
PyMCSCMDGP(pymc extra): plug a user-built PyMC causal model into badgers — structural nodes are observed RVs whose distribution parameters carry the structural functions; mechanism fitting is one jointpm.sample(full posterior kept), sampling and interventions run on the generic protocol with caller-provided rngs and ONE coherent posterior draw per cascade. Supported families:Normal,StudentT,Laplace(additive) andBernoulli,Poisson,Binomial,Categorical(numeric-encoded discrete). -
ClassSelectorpreset (badgers.core.selectors): per-class fixed-count sampling on a discrete column —ClassSelector([pos], class_column=pos, proportions={...}, rng=...)withposthe class column's int position andproportions={class_value: proportion}. Discovers the distinct values from the data (no manual class enumeration), raisesValueErroron class values absent from the column (typo protection), and selects exactlymin(int(prop * n_rows), n_rows_in_class)rows per listed class (partial mappings are legitimate: unlisted classes are kept). The class column is a column ofX, given as theclass_columnint position (selection reads onlyX). First subsampling consumer:RandomSamplingClassesGenerator. -
Per-node DGP protocol (
badgers.core.dgppackage):DGPprotocol andDGPMixin(topologicalfit/samplewalk) — backends implement exactly one operation,fit_node(node, parents, data), and exposeresponse(node, parent_values)anddraw_noise(node, rng, n);sample(rng, n)is tier 2 and no generator depends on it. Cascades compose each node through thestructural(node, parent_values, rng, n)hook (default: the additive compositionresponse + draw_noise; backends with non-additive mechanisms override it —GcmSCMDGPdelegates to dowhy'sevaluate). pick_parametershook (DGPMixin): called once at the top of the genericsample/intervenecascades; returns the DGP instance whose mechanisms drive the batch (ONE coherent parameter set per call). Default:self(deterministic backends fit exactly one parameter set).PyMCSCMDGPoverrides it to pin ONE coherent posterior draw from the caller's rng (a_ParameterDrawView): each cascade evaluates one coherent parameterization; the carrier itself evaluates draw 0. Posterior draws compose through the structure edits (an edited view rebinds the pinned base). The generator API is unchanged — backends are swappable transparently.- Value-level intervention
DGP.intervene(rows, nodes, values=None, *, rng=None)(Pearl surgery on given rows): fix the nodes atvalues(or at their current column values whenvalues=None), recompute ALL their descendants in topological order with FRESH noise; upstream and non-descendant columns are untouched; intervened nodes stay pinned (an intervened descendant is not recomputed over its pinned value). - Structure edits as methods on the DGP (read-only views; the base DGP
is never mutated; edits compose by delegation):
scale_edge(child, parent, k)(edge reweighting without coefficient access;calibrate(data)evaluates the variance budget and raisesValueErrorwhen infeasible),remove_edge(child, parent)(=scale_edge(..., 0.0)— SCM-surgery view),exogenize(node)(make the node a root: incoming edges removed from the view's graph),set_form(node, response_fn)(replace one node's functional form with a user closure; noise law inherited — pending Q2).add_edge/reverse_edgeraiseNotImplementedErrorpending the training-data-retention decision (A3). exogenizeis INTERVENTION-ONLY (documented contract): designed for the pin-then-intervene flow (the generator writes the node's column; the cascade never recomputes a root).sample()on an exogenized DGP draws the node as0 + N(0, residual_std)— NOT the real marginal; the empirical-resampling fix is deferred (it would require training-data retention).CausalOutlierPropagationGeneratornow propagates through the shared DGP machinery (internal change): the hand-rolled per-call lstsq forward pass is replaced by a fitted DGP (GcmSCMDGPfrom a dowhy-gcm SCM) +dgp.intervene(outliers, perturbation_nodes, rng=...)— uniform Pearl surgery semantics with the other causal generators. Behavior note: a degenerate zero-residual fitted mechanism now contributes no noise (the old0.1*col_stdfallback is gone).- Causal defect family (four generators sharing
CausalGenerator, an explicitfit(X, graph, column_mapping)phase followed bygenerate(...)withselector=row selection):CausalDriftGenerator(single systematic shift, recomputed downstream),CausalNoiseGenerator(per-edge noise injection,noise_std),CausalImbalanceGenerator(quantile-range truncation (pure subset, descendants untouched) or explicitselector=— selected rows dropped),CausalMissingnessGenerator(m-graph missingness withany/allpropagation, optional observation-graph derivation). -
pymcoptional extra (renamed fromcausal;pip install "badgers[pymc]") — only needed for building the PyMC SCM passed toPyMCSCMDGP. -
TopQuantileSelectorpreset (badgers.core.selectors): selects rows at or above thequantileof a column — self mode (own column) or driver mode (a driver column, optionally broadcast to all selected variables). Deterministic (no rng); thresholds computed on the called rows or on a separatestatsrow set (for generators realizing expanded rows against the original columns' statistics). All intensity parameters are now CONSTRUCTOR arguments of the presets (rate=forBernoulliSelector/ExactCountSelector,quantile=forTopQuantileSelector);__call__(X)takes no intensity kwarg, so an intensity sweep means constructing a new selector. Thequantileis the direct quantile of the column (quantile=0.75= everything at or above Q3).AlwaysSelectorwas added (the all-rows preset, theselector=Nonedefault of the causal generators). -
CausalImbalanceGeneratornow accepts an explicitselector=(additive; the oldquantile_rangerule remains theselector=Nonedefault): the rows SELECTED by the selector are dropped (subsampling polarity, union over the selector's variables);perturbation_nodes/quantile_rangeare ignored when a selector is given, andperturbation_nodesis no longer required in selector mode. DecompositionAndOutlierGeneratorrenamed toStructuredOutlierGenerator(old name kept as a backwards-compatible alias), with a new optionalpreprocessor=constructor argument: any sklearn pipeline-like object withtransform/inverse_transformcan replace the built-in standardization + decomposition — preprocessing is no longer hardcoded inside the generator.
Removed
- Seaborn is no longer a dependency of the
devextra. All tutorial/dev notebooks use plain matplotlib for visualizations. OlsDGP,DowhyGcmDGPandPyMCDGPDGP backends are removed. The only DGP backends are now the two SCM adapters (badgers.core.dgp.backends.GcmSCMDGPandPyMCSCMDGP): build the structural causal model with the wrapped library (dowhy-gcm or PyMC), then wrap it —GcmSCMDGP(scm)/PyMCSCMDGP(pymc_model). Seedocs/tutorials/dev/Using-DGPs.ipynb.edits.rebuildandedits.reversed_graphare removed (and with themCausalGenerator.rebuild_with_graph). Re-estimation after a structural modification is out of scope: build the modified SCM with the wrapped library's own API, fit it, and wrap it again with the appropriate adapter.
Changed
- Direct outliers generators delegate to the OOD samplers
(internal).
ZScoreSamplingGenerator,HypersphereSamplingGeneratorandHyperCubeSamplingno longer inline their value production; they delegate toZScoreSampler,HypersphereSamplerandUniformOutOfDistributionSamplerfrombadgers.core.sampling, making it the single source of out-of-distribution sampling math.UniformOutOfDistributionSamplernow acceptsexpansion=0(the exact data bounding box; previously it requiredexpansion > 0). One semantic behavior change: for degenerate (constant) columns, outliers are now pinned at exactly the constant value (bounding-box semantics of a zero range), where the oldMinMaxScalerround-trip spread them around/above the constant (e.g. constant 5.0:expansion=0.1→ [4.949, 5.936];expansion=0→ [5.041, 5.863], above the data max). Same seeds produce the same outliers up to floating-point ordering (z-score and hypersphere identical within ±1 ulp of the old scaler-based path; the hypercube's uniform draw is unchanged, values may shift by ±1 ulp on the new arithmetic path). Publicgeneratesignatures are unchanged. - Causal generators require an explicit DGP (hard error when
missing). There is no implicit default backend: every causal defect
generator raises when no
dgp=is supplied to theCausalGeneratorconstructor. Build a DGP from the library of choice (dowhy-gcm or PyMC) and wrap it withGcmSCMDGP/PyMCSCMDGP. - Drift tutorials and dev notebooks: extensive fixes — collapsed summary
tables rebuilt, garbled prose fixed, one seeded rng per notebook
(
gen = Generator()naming,seed = 0, no globalnp.random), verification cells asserting the documented behavior (noise variance, constant drift shift, edge correlation/std preservation, class counts, NaN counts, repeat/drop lengths), and causal-outlier plots usingget_outliers_labels(). fired()renamed toget_selected_fields()(breaking rename, no shim). Same contract and behavior — narrow bool field(n_rows, len(targets))(columni↔ the selector'si-th target),selector=None→ all-true WITHOUT consuming rng draws, single realization rulerng.random(n) < spec. The selector layer migrates to the select/selected vocabulary in code and docstrings; the outliers plumbing module_firing.pybecomes_selection.pywith helpersselect_and_seed(wasfire_and_seed) anditer_selection_groups(wasiter_firing_groups). (commit2621df9)
Documentation
- New "Understanding badgers" concepts notebook
(
tutorials/concepts/Badgers-Concepts.ipynb): a step-by-step introduction — module structure, generator instantiation with an rng,generate(), shape rules (appending / selection / subsampling), the in-place mutation contract ("passX.copy()"), the X-only contract (target-like data as columns ofX), selectors, DGP backends and pipelines. - Tabular tutorials merged per deficit (one notebook per deficit
instead of causal/non-causal pairs):
Outliers.ipynb,Noise.ipynb,Missingness.ipynb,Imbalance.ipynb,Drift.ipynb. Each has Part A (acting on the observation process — no DGP) and Part B (acting on the DGP). Visualizations use explicitclsclass vectors andlabelsdeficit vectors (noy/ytplaceholders); the outlier tutorial was substantially shortened (the causal part no longer repeats the concepts notebook).
Fixed
AdditiveLinearTrendGeneratorcrashes whenstart=None(the default). The randomly drawn start was a float (uniform(0, int(0.5 * len(X)))) and hit numpy slicing →TypeError: 'float' object cannot be interpreted as an integer. The draw is now cast toint; the signature documentsstart=None(random start in the first half of the signal) andend=None(trend lasts until the end).generate_random_patterns_indicescapacity guard off by one. The pattern-position helper (RandomlySpacedLinearTrends,LocalNoiseGenerator,RandomlySpacedLocalNoiseGenerator, value-drop generators) passed a signal of size 11 with 2 patterns of max width 6 — then crashed insiderng.integerswithhigh <= 0. The guard is nowmax_width_patterns * n_patterns <= signal_size(each equal-length segment must hold one full-width pattern).RandomChangeInMeanGeneratorcould draw duplicate changepoint indices. All indices were drawn independently, so one position could appear twice in the reportedchangepointslist (verified: 14 duplicates in 40 draws) and the realized level jump at a repeated position was the sum of two draws. Indices are now drawn distinct (sampling without replacement) and reported sorted by index; each reported change is the realized level jump at that position (documented in the docstring). Same-seed outputs change (different draw mechanism).GaussianNoiseClassesGeneratorno longer silently skips unconfigured classes (contract change). A selected class value absent fromnoise_std_per_classnow raisesValueError(previously it silently received no noise — documented-and-tested partial mappings are gone; pass a zero std explicitly to keep a class noise-free). The dict keys are normalized against the class column's dtype: plain-int keys work for a float-encoded class column, and keys of a wrong kind (e.g. strings for numeric classes) raise instead of silently disabling the noise.- The per-class drift generators take the class column from the selector
(contract change).
RandomShiftClassesGeneratorandFixedShiftClassesGeneratorno longer accept aclass_column=generate argument — the selector owns the class information. The class column is resolved from the selector'sclass_column(ClassSelector, typical); a selector that exposes no class column and an out-of-range position both raiseValueError(the previous "all selected rows treated as one class" fallback is gone). The dictshift_std/shiftknobs follow the same contract asnoise_std_per_class: a selected class value absent from the dict raises (no more silent skip), keys are dtype-normalized (plain-int keys work for a float-encoded class column), wrong-kind keys raise. selector=Nonenow means "the relevant default selector" where one exists. The three all-rows generators —AdditiveNoiseGenerator,RandomShiftGeneratorandFixedShiftGenerator— substituteAlwaysSelector(columns=range(X.shape[1]))(noise/shift on every row, the previous implicit behavior, now an explicit documented default). Where no relevant default exists,Nonestill raises with a rationale: the per-class drift/noise generators need a selector that owns the class column, the causal generators need the user's intervention targets, andRandomSamplingTargetsGeneratorneeds the user's deficit scores;RandomSamplingClassesGeneratorkeeps its documented no-op ("selects nowhere"). Realization change inget_selected_fields(): constant scalar specs (exactly0.0/1.0) are realized deterministically WITHOUT arngdraw (fractional scalars and bool arrays still draw). Results per selector are identical, but same-seed downstream draws shift for callers whose selectors produce constant scalar specs (e.g. the newAlwaysSelectordefaults).FixedShiftGenerator.generaterenamesshift_std→shift(breaking). The deterministic generator's parameter now matches the paired per-class generator (FixedShiftClassesGenerator.generate(X, shift=...)); the random counterparts keepshift_stdcorrectly. No deprecation shim — passingshift_std=raises aTypeErrorlike any unknown keyword.- Parameter validation switched from bare
asserttoraise ValueError(contract change). All 30 bareassertvalidations across the outliers, missingness, text, graph, time-series generators and core utils (e.g.assert expansion >= 0,assert 0 <= percentage_missing <= 1,assert bins > 0,assert size > 0) are now explicitValueErrors with descriptive messages — bare asserts silently vanish underpython -O. Code catchingAssertionErrorat these sites must catchValueErrorinstead (8 tests updated). - Stale "capped at len(X)" docstrings corrected. The outlier generators'
docstrings claimed a requested outlier count is capped at the dataset size
where the code actually raises (also true of the internal
CountSelectorfallback). Wording is now "The count must not exceedlen(X)(a larger count raises)" —distribution_sampling(3 sites), the low-density trio (3 sites +_resolve_count_and_targets) andinstance_sampling. - Scalar
slopein the time-series trend generators is broadcast to every column (bug fix).AdditiveLinearTrendGeneratorandGlobalAdditiveLinearTrendGeneratorpreviously crashed with a scalar slope on non-square X (ValueError: could not broadcast ... (n,) into (n,2)) and silently applied the ramp along COLUMNS on square X (n == n_cols) — a transposed trend. A scalar slope now applies the same per-row ramp to every column (the documented behavior, consistent with the array-slope path, whose outputs are unchanged). - The five hand-copied causal
get_*_labels()getters are unified (internal refactor, no behavior change): drift, missingness, imbalance and both noise generators now delegate to one shared helper on the causal base (_get_deficit_labels); the error message is derived from each generator'sdeficit_name. SwapLettersGeneratorno longer mutates the input list. The other text generators return a new list, so did this one — the returned list is a copy and the input words are left unchanged (same-seed outputs are identical to before).- Graph missingness generators raise
TypeErrorfor non-nx.Graphinput (wasNotImplementedError— nothing is "not implemented yet", the input type is wrong). - Time-series
MissingAtRandomGeneratorrenamed toMissingCompletelyAtRandomGenerator(breaking rename, no shim): the generator is MCAR — values are set to NaN uniformly at random, with no dependence on any observed column (the missingness mechanism of the tabular generators is already calledMissingCompletelyAtRandom). The benchmark registration is renamed accordingly. - Docstring hygiene sweep. The drift base-class docstring no longer
claims to "add noise" (it shifts data); the time-series
MissingValuesGeneratordocstring no longer claims to generate "point outliers" (it generates missing values); the "thant" typo in the two-dimensional-values error message of utils.py is fixed; the obscurefor...elseingenerate_random_patterns_indices(its else-body always ran after loop completion and read the leaked loop variable) is restructured to plain code with the same draw order and outputs; theimport warningsinside_column_stdsmoved to module top.
dev_0_0_14 (released)
Added
CausalOutlierPropagationGenerator: New tabular generator that propagates outliers along causal-graph edges. First generator to rely on the causal-graph API.- Causal-graph utilities:
validate_dag,topological_order, and related helpers for working with causal graphs. - Complete tabular generator benchmark coverage: All 16 tabular generators are now registered in the benchmark framework (up from 7). Added registrations for
HistogramSamplingGenerator,ZScoreSamplingGenerator,LowDensitySamplingGenerator,HypersphereSamplingGenerator,GaussianNoiseClassesGenerator,RandomShiftClassesGenerator,RandomSamplingClassesGenerator, andRandomSamplingTargetsGenerator.
Changed
- Generators now output NumPy arrays (breaking): Tabular and time-series generators return NumPy arrays instead of pandas DataFrames/Series. Input handling still accepts pandas DataFrames/Series.
- Vectorized generators: Replaced per-row/per-column
.ilocloops with batched NumPy operations for improved performance. - Two-tier sampler API: The
samplerparameter is split intowithin_distribution_samplerandout_of_distribution_sampler. - Explicit
column_mappingparameter: The causal-graph API now requires an explicitcolumn_mappingparameter. - Benchmark outputs moved to
.benchmarks/: Results and baselines are now stored in.benchmarks/results/and.benchmarks/baselines/(gitignored) instead ofbenchmarks/results/andbenchmarks/baselines/.
Removed
node_to_indexparameter from the causal-graph API (replaced bycolumn_mapping).
dev_0_0_13 (released)
Changed
- Separated tests from benchmarks: Generator correctness tests now live in
tests/generators/as standalone pytest functions with fixtures, independent of the benchmark framework. Thebenchmarks/checks/directory andFunctionalCheckinfrastructure have been removed. Benchmarks now focus exclusively on performance measurement (time/memory). (9 commits:943e230..56fee5d)
Added
- Pytest fixtures for each data type:
tabular_small/tabular_large,time_series_sine/time_series_walk,graph_erdos_renyi,text_word_list - Test conftest files in
tests/generators/{tabular_data,time_series,graph,text}/
Removed
benchmarks/checks/directory and allFunctionalCheckinfrastructure--type functionaloption from benchmark CLIfunctional_checksparameter fromGeneratorBenchmarkmodel