API: preprocessing.outliers
skyulf.preprocessing.outliers
Outlier-handling nodes package.
Split from a single 539-LOC module into per-node files
_common.py — shared mask helpers (y filter, pandas mask apply) iqr.py — IQR zscore.py — ZScore winsorize.py — Winsorize manual_bounds.py — ManualBounds elliptic.py — EllipticEnvelope clip_values.py — ClipValues (fixed bounds, clips instead of dropping rows)
All public names are re-exported here so existing imports such as
from skyulf.preprocessing.outliers import IQRCalculator continue to work.
ClipValuesApplier
Bases: BaseApplier
Set values below lower to lower and above upper to upper.
Nulls stay null, other columns are untouched and every row is kept, so unlike ManualBounds this step also runs on scoring rows.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 | |
apply(X, y, params)
Clip the configured columns on the active engine; y passes through.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
109 110 111 112 113 114 115 | |
resolve_fitted_config(raw, state)
staticmethod
Bind inference configuration to this node's inspected fitted artifact.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
96 97 98 99 100 101 102 103 104 105 106 107 | |
validate_fitted_state(raw)
staticmethod
Inspect this node's supported saved state without fitting or applying data.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
91 92 93 94 | |
ClipValuesCalculator
Bases: BaseCalculator
Validate the configured bounds; nothing is learned from the data.
bounds maps a column to {"lower": ..., "upper": ...}; either side
may be omitted. Use Winsorize for bounds learned from percentiles.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 | |
fit(X, _y, config)
Return the validated bounds as the artifact.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
175 176 177 178 179 180 181 182 | |
infer_output_schema(input_schema, config)
Promote fractionally bounded columns to match both runtime engines.
Source code in skyulf-core/skyulf/preprocessing/outliers/clip_values.py
165 166 167 168 169 170 171 172 173 | |
EllipticEnvelopeApplier
Bases: BaseApplier
Drop rows that any fitted per-column EllipticEnvelope model flags as an outlier.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 | |
apply(X, y, params)
Filter rows through the fitted per-column models on the active engine.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
161 162 163 164 165 166 167 168 169 170 171 | |
inference_capability(state, *, engine)
staticmethod
Describe fitted row filtering with nonfinite rows preserved by the native applier.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
152 153 154 155 156 157 158 159 | |
validate_inference_state(raw)
staticmethod
Inspect saved envelopes and skipped-column metadata without estimating covariance.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | |
EllipticEnvelopeCalculator
Bases: BaseCalculator
Fit one univariate EllipticEnvelope per numeric column at the configured contamination.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 | |
fit(X, _y, config)
Fit the per-column envelopes, skipping columns with fewer than five usable samples.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 | |
infer_output_schema(input_schema, config)
Return the input schema unchanged: row filtering preserves the column set.
Source code in skyulf-core/skyulf/preprocessing/outliers/elliptic.py
203 204 205 206 207 208 | |
IQRApplier
Bases: BaseApplier
Drop rows outside the fitted per-column IQR bounds; null/NaN values are kept.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | |
apply(X, y, params)
Filter rows through the fitted per-column bounds on the active engine.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
48 49 50 51 52 53 54 55 56 57 58 | |
inference_capability(state, *, engine)
staticmethod
Describe saved row filtering; prediction still rejects any removed rows.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
39 40 41 42 43 44 45 46 | |
validate_inference_state(raw)
staticmethod
Inspect learned quartile limits without recomputing request statistics.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
29 30 31 32 33 34 35 36 37 | |
IQRCalculator
Bases: BaseCalculator
Fit per-column bounds at the quartiles widened by multiplier times the IQR.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | |
fit(X, _y, config)
Compute quartile bounds per numeric column; warn on empty or non-numeric ones.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | |
infer_output_schema(input_schema, config)
Return the input schema unchanged: row filtering preserves the column set.
Source code in skyulf-core/skyulf/preprocessing/outliers/iqr.py
107 108 109 110 111 112 | |
IsolationForestApplier
Bases: BaseApplier
Exclude rows using the saved joint forest, retaining unscored nonfinite rows.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | |
apply(X, y, params)
Apply one positional mask to features and targets on either native engine.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
82 83 84 85 86 87 88 89 | |
inference_capability(state, *, engine)
staticmethod
Declare local row filtering only after validating the fitted detector.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
72 73 74 75 76 77 78 79 80 | |
validate_inference_state(raw)
staticmethod
Inspect the fitted forest and its feature axes without running tree inference.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | |
IsolationForestCalculator
Bases: BaseCalculator
Fit one forest across selected numeric features, excluding the implicit target.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | |
fit(X, y, config)
Learn only from finite training rows; malformed options and fit errors propagate.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
114 115 116 117 118 119 120 121 | |
infer_output_schema(input_schema, config)
Row exclusion preserves input column names and dtypes.
Source code in skyulf-core/skyulf/preprocessing/outliers/isolation_forest.py
110 111 112 | |
LocalOutlierFactorApplier
Bases: BaseApplier
Exclude unseen rows using the fitted training population as the neighbor reference.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | |
apply(X, y, params)
Score new rows only, retaining nonfinite rows for explicit downstream handling.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
73 74 75 76 77 78 79 80 | |
inference_capability(state, *, engine)
staticmethod
Declare local row filtering against fixed learned neighbors.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
63 64 65 66 67 68 69 70 71 | |
validate_inference_state(raw)
staticmethod
Require novelty mode and internally consistent learned neighbor statistics.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | |
LocalOutlierFactorCalculator
Bases: BaseCalculator
Learn LOF training scores and novelty state with separate train/apply representations.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | |
fit(X, y, config)
Fit a saved detector; use fit_transform_train to obtain filtered training data.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
99 100 101 102 103 104 105 106 | |
fit_transform_train(df, config)
Filter training rows with negative_outlier_factor_, never novelty predict.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
108 109 110 111 112 113 114 | |
infer_output_schema(input_schema, config)
Filtering rows retains all input column names and dtypes.
Source code in skyulf-core/skyulf/preprocessing/outliers/local_outlier_factor.py
95 96 97 | |
ManualBoundsApplier
Bases: BaseApplier
Drop rows outside the user-specified per-column lower/upper bounds.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 | |
apply(X, y, params)
Filter rows through the configured bounds on the active engine.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
61 62 63 64 65 66 67 68 69 70 71 | |
inference_capability(state, *, engine)
staticmethod
Declare row-local filtering; prediction rejects requests that lose rows.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
53 54 55 56 57 58 59 | |
validate_inference_state(raw)
staticmethod
Inspect saved optional bounds without applying the row filter.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
46 47 48 49 50 51 | |
ManualBoundsCalculator
Bases: BaseCalculator
Package the user-configured bounds into an artifact without reading the data.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | |
fit(_X, _y, config)
Return the bounds config as-is; no statistics are computed.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
124 125 126 127 128 | |
infer_output_schema(input_schema, config)
Return the input schema unchanged: row filtering preserves the column set.
Source code in skyulf-core/skyulf/preprocessing/outliers/manual_bounds.py
117 118 119 120 121 122 | |
WinsorizeApplier
Bases: BaseApplier
Clip values to the fitted percentile bounds; rows are never removed.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | |
apply(X, y, params)
Clip X to the fitted per-column bounds; y passes through untouched.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
182 183 184 185 186 187 188 189 190 191 192 | |
inference_capability(state, *, engine)
staticmethod
Describe local row clipping with saved limits without admitting Spark workers.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
174 175 176 177 178 179 180 | |
validate_inference_state(raw)
staticmethod
Inspect learned bounds and metadata without recomputing request quantiles.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 | |
WinsorizeCalculator
Bases: BaseCalculator
Fit per-column clip bounds at the configured lower/upper percentiles.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | |
fit(X, _y, config)
Compute each column's lower/upper quantiles; warn on empty or non-numeric ones.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | |
infer_output_schema(input_schema, config)
Preserve integer columns and retain existing promotion for other selected types.
Source code in skyulf-core/skyulf/preprocessing/outliers/winsorize.py
240 241 242 243 244 245 246 247 248 249 250 | |
ZScoreApplier
Bases: BaseApplier
Drop rows whose |z| against the fitted mean/std exceeds threshold in any column.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | |
apply(X, y, params)
Filter rows through the fitted z-score threshold on the active engine.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
61 62 63 64 65 66 67 68 69 70 71 | |
inference_capability(state, *, engine)
staticmethod
Describe saved row filtering without bypassing the prediction row-count guard.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
52 53 54 55 56 57 58 59 | |
validate_inference_state(raw)
staticmethod
Inspect saved numeric statistics and native scalar threshold semantics.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
40 41 42 43 44 45 46 47 48 49 50 | |
ZScoreCalculator
Bases: BaseCalculator
Fit per-column mean and population std (ddof=0) for z-score filtering.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |
fit(X, _y, config)
Record each column's mean/std; warn on empty or zero-variance columns.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | |
infer_output_schema(input_schema, config)
Return the input schema unchanged: row filtering preserves the column set.
Source code in skyulf-core/skyulf/preprocessing/outliers/zscore.py
124 125 126 127 128 129 | |