Technical Opinion

Encouraging Excellence from Australian High School students who will Graduate with SAT+AP

Author: David Pratten http://www.davidpratten.com 30-Apr-2006

Introduction
A significant number of Australian students seek entry to NSW & ACT university courses through the University Admissions Centre (UAC) on the basis of the American College Board's Scholastic Aptitude Test 1 (SAT) and Advanced Placement tests (AP). For these students the UAC will convert a SAT+AP score into a NSW & ACT University Admission Index (UAI) score.

Readers will find helpful background to this discussion in a prior technical report by the author titled "NSW and ACT school leavers and the conversion of SAT1 to an equivalent UAI" . The earlier technical report describes the current situation but does not give any recommendations for revision of the current schedule.

As noted in the prior technical report, there appear to be anomalies in the present conversion schedule. The effect of these apparent anomalies is to make it difficult for students to enter NSW & ACT universities on the basis of SAT results. This technical opinion describes a way of making the conversion fairer for all concerned.

This technical opinion addresses the following topics in order:

• A generalised conversion function and its fit to the current schedule
• The role of the AP aggregate,
• A proposed conversion schedule,
• Sensitivity analysis.

A Generalised Conversion Function
The current SAT conversion schedule may be generalised through a function with the following variables:

• a SAT score (sat),
• AP Aggregate band (apband),
• SAT Mean (mean),
• Standard Deviation(sd) and
• Participation Rate(pr).

Here is the generalised conversion function:

Generalised Conversion Function(sat,apband)
= (100-pr)/2 + normsdist((sat-mean)/sd)*(100-(100-pr)/2) + APdiscount(apband)

The normsdist() function is the standard normal cumulative distribution function which allows us to convert between the normal distribution of the SAT and the equi-percentile distribution of the UAI.

Based on experimental curve fitting (using Excel), the current conversion schedule is a good fit to the SAT, if the SAT has the following distribution:

• SAT mean=1458,
• SAT standard deviation=528, and
• SAT participation rate=30%.

The following chart shows how closely the current schedule fits that distribution:

However, based on the discussion in the earlier report, the SAT distribution is closer to the following values:

• SAT mean = 1500,
• SAT standard deviation= 300, and
• SAT participation rate= 35%.

Factoring in AP test aggregate
The aggregate score of student's AP tests provides a means for students to demonstrate their readiness for Australian university studies beyond a baseline SAT achievement. The current conversion schedule discounts the awarded UAI depending on the student's AP test aggregate. It does this in three bands: aggregate 0-5, aggregate of 6-8 and aggregate of 9-20. The current schedule's handling of the AP aggregate may be modelled as a discount varing between 0 and 2 for the three bands:

APdiscount(AP9-20) = 0
APdiscount(AP6-8) = -1
APdiscount(AP0-5) = -2

Unfortunately, this arrangement acts as a disincentive for Australian students to perform well in AP tests as an Australian student may currently achieve the best possible outcome by achieving just three "C" grades in three AP tests.

This technical opinion recommends that the AP aggregate be assessed based on 4 bands: 0-5, 6-8, 9-14, and 15-20. Under this arrangement the student would need to achieve 3 "A" grades or 3 "B" grades and a "C" grade to achieve the best possible outcome.

In the current conversion schedule, on average, stepping from one AP aggregate band to the next improves your UAI outcome by 1 UAI point. This technical report follows the same approach. The conversion schedule given below assumes the following discounts for the proposed 4 AP aggregate bands:

APdiscount(AP15-20) = 0
APdiscount(AP9-14) = -1
APdiscount(AP6-8) = -2
APdiscount(AP0-5) = -3


A Proposed Conversion Schedule
Based on conservative assumptions about the SAT distribution (m=1500, sd=300, and participation rate=35%) and handling the AP aggregate as explained, the generalised conversion function gives the following proposed conversion schedule.

Proposed SAT to UAI Conversion Schedule
for m=1500, sd=300, and participation rate=35%

Table 1.

Here is a visualisation of the portion of this conversion schedule relevant for university entrance.

Chart 1.

Naturally, once the College Board publishes exact mean, standard deviation and participation rates for the new SAT, the generalised conversion function can be used to update the proposed schedule.

Sensitivity Analysis
Because the distribution of the SAT changes year on year, there will always be a measure of uncertainty surrounding the statistical measures of the SAT test. The three main variables that conversion relies on are:

• SAT mean,
• SAT standard deviation, and
• SAT participation rate.

The general conversion function is not equally sensitive to variation in these parameters. To which parameter is the conversion most sensitive? Or put another way which parameter is it most important to get right?

Chart 2. shows the impact of variations of +/- 10% around the assumed mean of 1500, (in this and subsequent charts, an APdiscount of 0 is assumed):

Taking sat1900 as our indicative score, for our assumed mean(1500) the awarded UAI is 93.8. However, if the real mean of the SAT is 1650 (instead of 1500) then the appropriate equivalent UAI is 86.3! On the other hand if the real mean of the SAT is 1350 then the equivalent UAI would be 97.7! The conversion function is quite sensitive to variations in the mean of the SAT.

If the mean of the SAT is actually 1458, as suggested by the current conversion schedule, then the sat1900 student should receive a UAI of 95.3 instead of 93.8!

Chart 3. shows the impact of variations of +/- 10% around the assumed standard deviation of 300:

The conversion function is comparatively insensitive to variation in the standard deviation.

Chart 5. shows the impact of variations of +/- 10% around the assumed American participation rate of 35%:

For our indicative student, a 10% variation in the participation rate makes almost no impact on the conversion function.

While it is important to use the most accurate statistical values of the SAT, this sensitivity analysis has shown that it is most critical that the correct mean be used.

Conclusion
The generalised conversion function is capable of explaining the assumptions of the current conversion schedule and is also capable of generating new conversion schedules that more closely reflect the underlying SAT statistical distribution.

A new schedule has the potential of appropriately rewarding the achievement of Australian students who attempt to enter university on the basis of SAT plus AP scores.

This report recommends the use of the generalised conversion function with the mean, standard deviation and participation rate data provided by the College Board.

The creation of a 4th AP aggregate band will be of benefit to Australian students, giving them a reward for extra effort expended in preparing for university entry. In addition, if students believe that the conversion schedule will faithfully convert their study efforts into an Australian UAI they will be encouraged to do their best.