| |
 |
| Paper: |
A Rule-Based Data Quality Startup Using PyCLIPS |
| Volume: |
394, Astronomical Data Analysis Software and Systems (ADASS) XVII |
| Page: |
723 |
| Authors: |
DuPlain, R.F.; Radziwill, N.M.; Shelton, A.L. |
| Abstract: |
A rule-based approach to data quality provides for efficient and extensible solutions in validating data sets. A working prototype proves that CLIPS, a trusted rules engine, can integrate with existing data processing libraries through PyCLIPS, resulting in a system which isolates data quality rules from programming logic to allow for parallel development and maintenance of rules and applications. The prototype demonstrates several benefits: developers can treat rules independently from application source code, a ruleset can accept new rules without modification of existing rules in the set, rules can provide value through conditional assertions which call on external tools, and even a very small rule set can point to real errors in telescope data. A rules engine which automates data quality validation with existing tool libraries could potentially lead to a substantial increase in operational availability even when faced with resource scarcity. |
|
|
|
|
|
