Difference between revisions of "Validation and Corrections"
LornaMorris (talk | contribs) (→Running the automated correction) |
LornaMorris (talk | contribs) (→Validation) |
||
| (29 intermediate revisions by 2 users not shown) | |||
| Line 17: | Line 17: | ||
| − | + | The left-hand panel shows a list of validation errors and in the main editor window the data file is displayed. Clicking on any individual validation error in the left-hand panel takes the user to the corresponding error in the data file. Errors are marked with an red error icon within the left-hand margin. It is possible to make manual edits to the file to fix these errors, but when there are so many errors within a file this would be labour intensive. It the next step (the automated correction) we show how these errors can be fixed automatically using the reBiND correction software. | |
== Running the automated correction == | == Running the automated correction == | ||
| Line 27: | Line 27: | ||
| − | A drop-down menu gives a list of available configuration files. The first correction configuration ('default correction') is suitable for most ABCD files. Alternative corrections can be uploaded by the administrator to run different automated corrections | + | A drop-down menu gives a list of available configuration files. The first correction configuration ('default correction') is suitable for most ABCD files. Alternative corrections can be uploaded by the administrator to run different automated corrections This could depend on - for example - what sort of errors have been seen in the data file (in the validation step) or whether a different XML file has been used instead of the default ABCD data. |
| − | |||
| − | |||
| − | |||
| + | After clicking on 'Start Correction' a throbber appears as the correction modules (specified in the configuration file) are run. When the correction is complete a report is generated (see the following screenshot): | ||
| + | [[File:Correction_output_report.png|border]] | ||
| − | + | ||
| + | The output shows a link to the original data file, a link to an XML version of the report and a tabular view of the report showing the number of each type of correction made. The level of 'info', 'warning' and 'error' are used to indicate the effect of the change as follows: | ||
| + | |||
| + | * info - flags any minor change to the data where no problems are expected from the change. | ||
| + | * warning - flags a change where it is uncertain that the new value is correct and it should be checked by the content administrator. | ||
| + | * error - flags a a problem that could not be corrected and results in the file being invalid according to the associated schema. | ||
| + | |||
| + | |||
| + | The report indicates several changes were made, including one change of a year date to an ISO DateTime and 1640 changes of abcd:LowerValue. These changes are both of type 'Element Text Replacer', where a standard pattern in the text is replaced with another. There were also 201 changes to remove unnecessary empty elements. There is another type of change called an 'Element Renamer' which renames incorrectly named elements within the XML file. Clicking back on the browser and then opening the reBiND editor by clicking the action 'Edit' under the data file allows the user to review these results from the automated correction. See the next sections for details on reviewing the corrected data file. | ||
Latest revision as of 02:07, 19 November 2014
Validation and Correction
Once the XML data file has been uploaded into the reBiND system the user can validate and perform automated and manual corrections to the file before publishing. Publishing the data makes it available to the public search interface and also to biodiversity networks, such as GBIF.
Validation
The figure below shows the result of clicking on the validation action for the file reBiND_Puffinus.xml. When the validation is running the information screen opens and displays a throbber while the file is being validated. In the screenshot the validation is complete and the screen shows the result - that there are 600 errors in the file.
After validation is complete it is possible to review the validation results in detail in the reBiND editor (a modified version of the eXide editor which comes bundled with the eXist software). To open this editor the user should click on the 'Edit' button in the list of actions below the data file (in this case reBiND_Puffinus.xml). This opens the data file in the editor - a screenshot of this is shown below:
The left-hand panel shows a list of validation errors and in the main editor window the data file is displayed. Clicking on any individual validation error in the left-hand panel takes the user to the corresponding error in the data file. Errors are marked with an red error icon within the left-hand margin. It is possible to make manual edits to the file to fix these errors, but when there are so many errors within a file this would be labour intensive. It the next step (the automated correction) we show how these errors can be fixed automatically using the reBiND correction software.
Running the automated correction
'Start Correction' is the final action in the list below the data file. Clicking on this link takes the user to the following page:
A drop-down menu gives a list of available configuration files. The first correction configuration ('default correction') is suitable for most ABCD files. Alternative corrections can be uploaded by the administrator to run different automated corrections This could depend on - for example - what sort of errors have been seen in the data file (in the validation step) or whether a different XML file has been used instead of the default ABCD data.
After clicking on 'Start Correction' a throbber appears as the correction modules (specified in the configuration file) are run. When the correction is complete a report is generated (see the following screenshot):
The output shows a link to the original data file, a link to an XML version of the report and a tabular view of the report showing the number of each type of correction made. The level of 'info', 'warning' and 'error' are used to indicate the effect of the change as follows:
- info - flags any minor change to the data where no problems are expected from the change.
- warning - flags a change where it is uncertain that the new value is correct and it should be checked by the content administrator.
- error - flags a a problem that could not be corrected and results in the file being invalid according to the associated schema.
The report indicates several changes were made, including one change of a year date to an ISO DateTime and 1640 changes of abcd:LowerValue. These changes are both of type 'Element Text Replacer', where a standard pattern in the text is replaced with another. There were also 201 changes to remove unnecessary empty elements. There is another type of change called an 'Element Renamer' which renames incorrectly named elements within the XML file. Clicking back on the browser and then opening the reBiND editor by clicking the action 'Edit' under the data file allows the user to review these results from the automated correction. See the next sections for details on reviewing the corrected data file.
