At source validations

The NCMP system validates data as it is entered. For each record the system checks that all mandatory fields have been populated and that each field contains valid data. Records with missing mandatory fields or invalid data cannot be saved (classified as “rejections”). Records with improbable fields (e.g. extreme measurements) can be saved but will generate “warnings” that the data provider will need to confirm before they can finalise their submission.

Data can be entered into the NCMP system in two ways and validation can differ depending on the approach used. The two ways of entering data follow along with information on how rejections and warnings are handled

1. Individual record data entry

Data are typed directly into the system by users

a. Rejections: it will not be possible to save the record unless all entered data are valid and mandatory fields have been populated.

b. Warnings: it will be possible to save records with warnings but all warnings must be confirmed by the data provider before the submission can be finalised

2. Multiple record data entry

Data are uploaded into the system via a CSV file.

a. Rejections: records containing invalid data and/or missing mandatory fields will be flagged to users for correction before their upload can be completed.

b. Warnings: records with warnings will be flagged to users and can be confirmed or corrected before the upload is completed, or they can be uploaded with the warnings. All warnings must be confirmed by the data provider before the submission can be finalised.

The following validations are performed by the NCMP system:

1. URN (DfE school number) Rejection

This is a mandatory field and must be populated with a valid URN

• individual data entry: data item is automatically populated by the system based on the school grid the user is adding data to
• uploaded data: URN must belong to one of the schools on the user’s schools list

2. Date of Birth (DoB) Rejection

This is a mandatory field and must be populated with a valid DoB. Valid DoB ranges are set each year as part of collection year set up. These ranges ensure that only children aged 4, 5, 10 or 11 years of age can be entered into the system. As an example, the ranges set for the 2014/15 collection year were: 01/09/2009 to 01/09/2010 for Reception and 01/09/2003 to 01/09/2004 for Year 6. For details of the date ranges for the current school year, please refer to the Reference Data Excel file on the main NCMP guidance

• individual data entry: the user must select a valid DoB from dropdown menu
• uploaded data: each record must contain a valid DoB in dd/mm/yyyy format

3. Sex – Rejection

This is a mandatory field and must be populated.

• individual data entry: the user must select a sex from dropdown menu.
• uploaded data: each record must contain one of the following valid responses: M, F, m, f, MALE, FEMALE, male, female.

4. NHS number Rejection

This is not a mandatory field and can be blank. However, users are strongly encouraged to provide this data item since it will allow individual NCMP records to be linked such as a child’s reception and year 6 measurements. The proportion of records with a missing NHS number is shown in the data quality table (see later). Where a value is provided, the system checks that the NHS number is valid using the Modulus 11 algorithm

5. Ethnicity Rejection

This is not a mandatory field and can be blank although users are strongly encouraged to provide this data item since it is an important variable for analytical purposes into inequalities. The proportion of records with a missing ethnicity code is shown in the data quality table (see later). Where a code is provided it must be included in the codes list held in the NCMP system. Currently the NCMP System accepts 342 ethnic codes taken from the following sets of ethnicity codes: four-character DfE ethnicity codes; single-character NHS ethnicity codes; RIO ethnicity codes and SystmOne ethnicity codes. These are shown in the reference data tables on the NCMP system

6. Height and weight

There are two checks carried out

Rejection

Error ranges have been set for height, weight and BMI. If an entered height or weight falls outside the relevant range, or their combination produces a BMI that falls outside the BMI range, then the record will be rejected. The ranges have been set by analysing historical NCMP data. The following ranges are currently in use:

Warning 

Warning ranges have been set for height, weight and BMI. These ranges use standardised values (z-scores) which take child age and sex into account. Measurements resulting in z-scores outside the “warning” ranges will generate warning flags. It is possible to save records with warning flags but all warnings must be confirmed by the data provider before the submission can be finalised. The “warning” ranges currently in use are zscores lower than -3 to or higher than +4.  The standardised value is called a z-score and indicates how far, and in what direction, the measurement deviates from the average (mean) for that age and sex (see Annex A for more detail).

7. Date of Measurement (DoM)

There are two checks carried out

Rejection

Each year a DoM range is set as part of collection year set up. If an entered DoM falls outside the range then the record will be rejected. This ensures that the child has been measured within the correct academic year.  For example, the range used for the 2014/15 collection year was 01/09/2014 to 31/08/2015 (For details of the date ranges for the current school year, please refer to the Reference Data Excel file. 

Warning

If an entered DoM is a weekend day or in the month of August then a warning flag will be added to the record since schools are presumed to be closed on these days. It is possible to save records with warning flags but all warnings must be confirmed by the data provider before the submission can be finalised.

Individual data entry: the user must select a valid DoM from the dropdown menu.

Uploaded data: each record must contain a valid DoM in dd/mm/yyyy format.

8. Child postcode.

These checks are carried out on the child postcode.

Rejection

The postcode must be in a valid format. It is important to note that this check is just against the format of the postcode and not whether the postcode is actually valid as doing so would make the NCMP system run too slowly.  For example, LS99 9ZZ would pass the format check but it is not a valid postcode.  Therefore, all postcodes should be checked for validity before they are entered into the system. The following postcode formats are accepted by the system: A9 9ZZ; A99 9ZZ; AB9 9ZZ; AB99 9ZZ; A9C 9ZZ; AD9E 9ZZ

Warning

Child postcode is not a mandatory field and can be blank although users are strongly encouraged to provide this data item since it is required for analyses based on child residence. The proportion of records with a missing postcode is shown in the data quality table (see later).  Each record with a blank postcode field will generate a warning requiring confirmation from the submitting organisation before the submission can be finalised.   

Warning

Data providers are asked to confirm any child postcodes which are the same as the school postcode.  In other words, the data provider confirms that the child does live very near to the school and that the school postcode has not been submitted in error. 

9. Duplicate check - Rejection

The system currently defines duplicates as records within one school with the same: Pupil ref and/or NHS number and/or first name, surname, sex and DOB

• individual data entry: it is not possible to save any record defined as a duplicate. Therefore if a user attempts to add a record into a school’s pupil grid and enters the same pupil ref and/or NHS number and/or first name, surname, sex and DOB as another record in that school, then the record will be rejected.
• uploaded data:  when uploading records the system checks whether a record is “new” (i.e. not already held) or an “update” (i.e. record already held). The system will check the following fields in order, moving on to the next field if no data is held for the field being checked: NCMP system ID; Pupil Ref (the LA’s reference); NHS number; First Name, Last Name, Date of Birth, Sex. Where the system finds a record in the upload file matching one of these fields for a record in the system, and both records have the same URN, the system will update the record in the system. However, the system will not allow any matched records to be uploaded if they create a duplicate record. So if, for example, a record in the upload file has the same NCMP system ID as a record in the system but also has the same Pupil ref and/or NHS number
• Where 9 can be any single digit number
• A can be any letter except for Q, V or X.
• B can be any letter except for I, J or Z.
• C can be any letter except for I, L, M, N, O, P, Q, R, V, X, Y or Z
• D can be any letter except for I, J or Z.
• E can be any of A, B, E, H, M, N, P, R, V, W, X or Y
• Z can be any letter except for C, I, K, M, O or V
• and/or first name, surname, sex and DOB as a third record, then the record will be rejected

10. Duplicate measures warning

As well as the duplicate check described above, the system will also warn users if they enter exactly the same height and weight combination as the previous record. This may indicate a mistake when manually entering data where the person inputting the data has started to enter the data for another child, but has accidentally duplicated the measurement data for a child which they have already entered into the system.

11. Same height and weight entered - Rejection

It is extremely unlikely that a child would have the same height in cm and weight in kg and it is more likely that the height has been duplicated in the weight field by mistake or vice versa. The system will reject all records with an identical height and weight.

Data providers also have access to a summary data quality table throughout the collection period.  More details on this table are given in the following section as it is also used as part of the post deadline validation.