NSW and ACT school leavers and the conversion of SAT1 to an equivalent UAI.

David Pratten MACS, MACM  d23921@prattens.net

For followup to this report see also Technical Opinion - Encouraging Excellence from Australian High School students who will Graduate with SAT+AP

19 February 2006

 

Some NSW & ACT high school leavers graduate high school while overseas.  These students potentially enrich Australian university communities with their global outlook. Many of these students seek entry to NSW & ACT university courses through the University Admissions Centre (UAC) on the basis of the American College Board's Scholastic Aptitude Test 1 (SAT1).

 

This report examines the nature of the SAT1 and its sections, their means, standard deviations and percentiles, and then compares SAT1 scores with NSW & ACT University Admission Index (UAI) scores.

 

UAC affiliated tertiary institutions assess SAT1 scores on the basis of a Common Assessment Schedule that appears to create barriers to university entry for NSW & ACT students who graduate from high school while overseas. These students appear to be penalised by up to 9 UAI points in the process of converting a SAT1 score to a UAI score.

 

After examining the Common Assessment Schedule and some of its effects on Australian students, this report finds good reasons for a review of the schedule and expresses the hope that NSW & ACT students with international experience and a global outlook will not be excluded from our campuses by its use.

 

Contents

NSW and ACT school leavers and the conversion of SAT1 to an equivalent UAI... 1 Contents...................................................................................................... 2 NSW & ACT School leavers with SAT1 qualifications....................................... 2 SAT1............................................................................................................ 3 SAT1 mean and standard deviation............................................................... 3 SAT1 Percentiles........................................................................................... 5 University Admission Index (UAI).................................................................. 5 Converting a SAT1 score to a UAI score......................................................... 6 Cohort Equivalence.................................................................................... 6 Conversion................................................................................................ 7 Common Assessment of Qualifications........................................................... 8 Impact of the Common Assessment Schedule............................................... 10 How is the boost and penalty distributed?.................................................... 11 Conclusion.................................................................................................. 12 References................................................................................................. 13 Copyright................................................................................................... 13

 

NSW & ACT School leavers with SAT1 qualifications

NSW & ACT students complete high school overseas for many reasons including parental work placement and student exchange.  The Australian university community is enriched by these students. Australia's international competitiveness is based on quality skills and global outlook. Students with life experience outside Australia, often knowing another world language, bring with them experience in navigating cross cultural situations and a world encompassing perspective.

 

While many of these students complete Australian high school qualifications by home based distance education, others do not. These students perform better with the socialisation provided by local high schools compared to home based study. US schools and many International Schools around the world prepare students to sit the SAT1 tests and, at graduation, provide students with a High School diploma. With these qualifications[1], NSW & ACT school leavers may enter participating universities through the University Admissions Centre (UAC).

 

What is the SAT1 test?

SAT1

The United States of America Scholastic Aptitude Test (SAT) 1 is a tertiary entrance test administered multiple times each year by the College Board and from 2005 onwards consists of three sections; Verbal, Mathematics and Writing.  Each section is marked in a range of 200-800.  The College board characterises the scores of each section as having an approximate normal distribution and publish the mean and standard deviation. The writing section was introduced in 2005 and the mean and standard deviations for the writing section will not be available until September 2006.

 

The SAT1 has a participation rate among graduating year 12 students of approximately 50% (IDOE 2005). Approximately 70% of American high school students graduate from high school (Greene 2002). This means that the SAT1 cohort represents 35% of all high school starters, the ones who aspire to enter university.

 

College Board (p7 2005)  show the following section mean and standard deviations.

 

Section	Mean	Standard Deviation
Verbal	508	113
Maths	520	115

Table 1.

 

From these actual mean and standard deviations, we can infer that the SAT sections are designed around a target mean of 500 and standard deviation of 100.  That is, the section score range of 200-800 represent 3 standard deviations either side of the mean.

 

The absence of public data for the mean and standard deviation of the Writing section is not critical for our present purposes. Any mean for the Writing section within the range 450-550 and standard deviation in the range of 50-150 may be chosen without affecting the conclusions of this report. 

SAT1 mean and standard deviation

For the purpose of calculating SAT1 mean and standard deviation, we will use the following means and standard deviations for the sections.

 

Section	Mean	Standard Deviation
Verbal	508	113
Maths	520	115
Writing	500*	100*

Table 2. * mean and standard deviation are estimates.

 

Combining the student's 3 section scores together gives the student a SAT1 score out of 2400. We may characterise the mean of the combined score is the sum of the 3 means, 1528, for our purposes, let's call it 1500.

 

With the information presented in  (College Board 2005) it is not possible to give a definite value for the SAT1 standard deviation. It is possible, however to set upper and lower bounds for the SAT1 standard deviation. The upper bound occurs when we assume that the scores of the three sections are fully dependent, in that case, the standard deviation would be the sum of standard deviations, that is, 328.  The lower bound occurs when we assume that the three sections are independent variables and then the standard deviation would be the square root of the sum of squares of standard deviations, that is, 190.

 

Lowering the assumed standard deviation towards 190 strengthens the observations of this report, so let's use a conservative 300 as our assumed SAT1 standard deviation.

 

	Mean	Standard Deviation
SAT1	1500*	300*

Table 3. * mean and standard deviation are estimates.

 

Because of a lack of public data on SAT1 mean and standard deviation, significant space in this report has been devoted just to deriving likely values. While the exact numbers will change, the conclusions of this report are not sensitive to significant variation in these assumed values. 

SAT1 Percentiles

In order to compare SAT1s with UAI,s we need to be able to calculate the percentiles associated with different SAT1 scores.  Using the standard normal cumulative distribution function we can construct the graph in Chart 1.

 

Chart 1.

 

Based on this chart, we see the characteristic bunching of scores around the mean (1500) of normal distributions. As well, as expected, practically all students score within +/- 3 standard deviations of the mean.

 

A student scoring 1100 on the SAT1 is a bottom 10% student, while a student scoring 1900 is a top 10% student.

 

Now, let us turn our attention to the NSW & ACT UAI.

University Admission Index (UAI)

The NSW & ACT University Admission Index is a number between 0 and 100 on the basis of which students may be competitively admitted to participating Australian tertiary institutions.  The number represents the percentile ranking of the student's achievement compared to all students who started high school in Australia in the same year. (NSW Vice-Chancellors Conference 1998 p5)  

 

While UAIs are awarded to approximately 75% of those who started high school in the same year (ABS 2004), the UAI's cohort is all students irrespective of whether they completed high school or not. 

 

Unlike the SAT1, the UAI has, by definition, a uniform distribution.  It is uniform if we include all students who started high school together.  Because UAI scores are not actually awarded to all of these students, the uniformity is limited to UAI25 and above.  In practice UAI scores less than 30 are reported as "30 or less" (Technical Committee on Scaling 2005 p5)

Converting a SAT1 score to a UAI score

In order to convert scores, we need to make an assumption about the relative abilities of the cohorts of students sitting the SAT1 and the UAI.

 

Cohort Equivalence

Following (NSW Vice-Chancellors Conference 1998) p2, and broadening the definition of "state" to mean "nation state",  this report accepts "the assumption that the age cohorts from which each State’s candidature are drawn are equally able to undertake university study."

 

Following from this, if the SAT1, as well as the UAI, cohort was all students who started high school, then they could be directly compared.  However we have already seen that the participation rate of the SAT1 is approximately 35% of American students.  How then can we map SAT1 scores to UAIs?

 

We can observe that the SAT1 cohort is more able to undertake university study than the UAI cohort.  This follows from our earlier observation that the SAT1 cohort is the 35% of American high school starters who aspire to enter university, while the UAI cohort represents all Australian students who started high school.  The average SAT1 student will be more capable than the UAI50 student.

 

One simplistic approach to equating SAT1 to UAI scores would be to map all SAT1 scores to the range UAI65-UAI100.  This would be reasonable if we assumed that the best 35% of all American students sat the SAT1.  This assumption is hard to defend.

 

A second equally unsatisfactory approach would be to map all SAT1 scores to the range UAI 0 to UAI 100.  This would only be reasonable if the SAT1 cohort was all American students who started high school.

 

A third approach would be to say with (NSW Vice-Chancellors Conference 1998) p2 "The truth lies somewhere in between." and map the SAT1 range to  the range UAI 32.5 to UAI 100.  This splits the difference between the first two alternatives. This approach to cohort equivalence, while probably underselling the qualities of the university bound SAT1 cohort, it is a conservative assumption that reduces the size of the possible anomalies detailed below.

Conversion

On the basis of this third approach to cohort equivalence SAT1 scores may be converted to UAI's by using Chart 2 below.  Chart 2 maps the SAT1 range into the portion of the UAI above UAI32.5 You can see that the shape is similar to what as we saw in Chart 1.

Chart 2.

 

From this chart, we can see that a student who achieves an average SAT1 score (1500) would be awarded a UAI of 67.  A student scoring on the 10^th percentile of the SAT1 (1100), would be awarded UAI39.  And, a SAT1 1900 student who scores in the top 10% of the American cohort would be awarded UAI94.

 

But how is conversion actually done by the Universities Admissions Centre?

Common Assessment of Qualifications

The Universities Admissions Centre (NSW & ACT) assists participating Australian tertiary institutions by providing a Common Assessment Schedule (UAC 2005) to assist them with converting SAT1 scores to equivalent UAI scores.

 

The schedule associated with the SAT1 for 2005/6 is SR S5053-55.  The schedule has been produced on the basis of observed score distributions, or if that was not available, on the basis of expert judgment. It was accepted by decision of the Common Assessment Procedures Sub-committee on 04 April 2005. (UAC 2005).

 

The schedule contains the conversions for students who have, and have not, sat Advanced Placement tests as well as the SAT1. For the purposes of this report, any of the conversion columns could have been used without affecting the conclusions.  We will use the table for students who have not also sat for Advanced Placements tests.  It is as follows:

 

Table 4. Extracted from Common Assessment Schedule SR S5053-55

 

Table 4. may be visualised as the following chart:

 

Chart 3.

 

The College Board considers SAT1 scores to be normally distributed. Looking at the almost straight line chart above, we can see that the Common Assessment Schedule SR S5053-55 treats SAT1 scores as if they are have such a large standard deviation[2] that the characteristic "S" curve of the standard normal cumulative distribution function almost disappears.

 

Impact of the Common Assessment Schedule

The Common Assessment Schedule appears to boost the competitiveness of some SAT1 holders for entrance to participating Australian tertiary institutions, and appears to penalise others.

 

This can be seen by overlaying the SAT1 to UAI conversion graphs (Chart 2.) with the Common Assessment Schedules graph (Chart 3.)

 

Chart 4.

 

From this chart, we can see how the Common Assessment Schedule affects students.

 

A student who achieves an average SAT1 score (1500) will be awarded a UAI of 68, roughly the same as the UAI67 calculated above.  This confirms our earlier observation about the relative abilities of the average SAT1 student and the UAI50 student.

 

A student scoring on the 10^th percentile of the SAT1 (1100), does better with a 11 point boost to UAI 50 (from UAI39)!  Unfortunately, however our SAT1 1900 student will only be awarded a UAI of 85, a nearly 9 point penalty from the UAI94 calculated above.

 

The symmetry of the two curves above lends credibility to this report's approach to cohort equivalence.

 

How is the boost and penalty distributed?

From the few examples above, we can see that the effect of the Common Assessment Schedule's approach is not uniform.  Chart 5. shows the amount of the boost and penalty for students of differing SAT1 scores:

 

 

Chart 5.

 

From this chart we can see that the students who achieve SAT1 1100 receive the maximum boost of 11 UAI points and students with SAT1 1900 are penalised the hardest at almost -9 UAI points.  Students who score SAT1 1550 are neither boosted nor penalised.

 

The current Common Assessment Schedule appears to apply a boost to non-competitive university applicants and a penalty to those who are competitive.

 

The range of penalties exceeding 5 UAI points occurs right around the critical area of entrance cutoffs for NSW & ACT university courses. The Australian student who achieves a top 10% result in the SAT1 will be awarded a UAI of only 85 and may miss out on placement in a course as a result.    

Conclusion

While the door is open for NSW & ACT school leavers to enter tertiary courses on the basis of SAT1 qualifications, the door seems to be open only part way.

 

Based on conservative assumptions of cohort equivalence, this report has shown how the schedule gives a significant boost to the scores of non-competitive school leavers, and at the same time, makes it tough for NSW & ACT students to enter courses at UAC's participating universities on the basis of SAT1 qualifications. 

 

The top 10% Australian student who happens to study overseas will need to achieve a top 4% result in the more competitive SAT1 cohort to achieve a top 10% UAI outcome!

 

The prospective students who are most affected by this are the NSW & ACT school leavers with international experience, a global outlook, and who potentially enrich the university community.

 

Based on these concerns, there appears to be good reason for reviewing the appropriateness of the current Common Assessment Schedule for the SAT1.

 

Hopefully, future use of the schedule will not exclude NSW & ACT students with international experience and a global outlook.

References

ABS 2005, 4221.0 Schools, Australia , retrieved 11-Jan-2005 from http://www.abs.gov.au/Ausstats/abs@.nsf/Lookup/1E44BCDEF87BCA2FCA2568A9001393E7

 

College Board 2005,  2005 College-Bound Seniors,  retrieved 22-Dec-2005 from http://www.collegeboard.com/prod_downloads/about/news_info/cbsenior/yr2005/2005-college-bound-seniors.pdf

 

Greene 2002, High School Graduation Rates in the United States, retrieved 10-Jan-2006 from http://www.manhattan-institute.org/html/cr_baeo.htm

 

IDOE 2005, Indiana SAT scores rise 5 points, retrieved 10-Jan-2006 from http://www.doe.state.in.us/reed/newsr/2005/08-August/SAT083005.html

 

NSW Vice-Chancellors Conference 1998, The Universities Admission Index - A Technical Report, retrieved 10-Jan-2006 from http://www.uac.edu.au/pubs/pdf/uai_tech_rep.pdf   

 

Technical Committee On Scaling 2005, Report on the Scaling of the 2004 NSW Higher School Certificate retrieved 20-Feb-2006 from http://www.uac.edu.au/pubs/pdf/tsc-report-2004.pdf

 

UAC 2005, SR S5053-55 United States of America SAT1 (2005 and onwards) Common Assessment Schedule, UAC Pty Ltd

 

Copyright

David Pratten 2005, 2006  http://creativecommons.org/licenses/by-nc/2.1/au/