Probabilities of October NIO 3 Year El Nio

Probabilities of October NIÑO 3 Year El Niño Neutral La Niña 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 0, 5 1993 0. 24 N 40 60 0 0, 4 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Verification Example using ROC Diagram 1, 0 ROC curves 0, 9 0, 8 0, 7 Hit rate Observed 0, 6 0, 3 E = 0. 85 N = 0. 62 L = 0. 85 no skill 0, 2 0, 1 0, 00 0, 50 False-alarm rate 1, 00

Categorical Contingency Table Observed Forecast yes no yes hits false alarms forecast yes no misses correct rejections forecast no observed yes observed no total hit - event forecast to occur, and did occur miss - event forecast not to occur, but did occur false alarms - event forecast to occur, but did not occur correct rejections - event forecast not to occur, and did not occur Hits Rate hits / (hits + misses) False Alarm Rate - false alarms / (hits + false alarms)

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Observed Neutral: 10 Non Obs Neutral: 10 Observed La Nina: 5 Non Obs La Nina: 15

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>0 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>0 H= Correctly fcst El Ninos Observed El Ninos H =5/5=1. 0 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>0 H= Correctly fcst El Ninos Observed El Ninos H =5/5=1. 0 F= Wrongly fcst El Ninos Non Obs El Ninos F =15/15=1. 0

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>20 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>20 H= Correctly fcst El Ninos Observed El Ninos H =4/5=0. 8 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>20 H= Correctly fcst El Ninos Observed El Ninos H =4/5=0. 8 F= Wrongly fcst El Ninos Non Obs El Ninos F =5/15=0. 33

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>40 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>40 H= Correctly fcst El Ninos Observed El Ninos H =4/5=0. 8 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>40 H= Correctly fcst El Ninos Observed El Ninos H =4/5=0. 8 F= Wrongly fcst El Ninos Non Obs El Ninos F =3/15=0. 2

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>60 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>60 H= Correctly fcst El Ninos Observed El Ninos H =3/5=0. 6 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>60 H= Correctly fcst El Ninos Observed El Ninos H =3/5=0. 6 F= Wrongly fcst El Ninos Non Obs El Ninos F =0/15=0. 0

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>80 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>80 H= Correctly fcst El Ninos Observed El Ninos H =3/5=0. 6 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>80 H= Correctly fcst El Ninos Observed El Ninos H =3/5=0. 6 F= Wrongly fcst El Ninos Non Obs El Ninos F =0/15=0. 0

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>100 H= F= Correctly fcst El Ninos Observed El Ninos Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>100 H= Correctly fcst El Ninos Observed El Ninos H =2/5=0. 4 F= Wrongly fcst El Ninos Non Obs El Ninos

Probabilities of October NIÑO 3 Year Observed El Niño Neutral La Niña Observed El Ninos: 5 1981 -0. 23 N 0 60 40 1982 2. 07 E 100 0 0 Non Obs El Ninos: 15 1983 -0. 21 N 0 100 0 1984 -0. 84 L 0 80 20 1985 -0. 82 L 0 80 20 1986 0. 55 E 0 100 0 1987 1. 28 E 80 20 0 1988 -1. 62 L 0 60 40 1989 -0. 41 N 20 80 0 1990 -0. 10 N 40 40 20 1991 0. 62 E 40 60 0 1992 -0. 33 N 40 60 0 1993 0. 24 N 40 60 0 1994 0. 47 N 0 80 20 1995 -0. 86 L 0 60 40 1996 -0. 49 N 0 80 20 1997 3. 02 E 100 0 0 1998 -0. 71 N 0 20 80 1999 -1. 09 L 0 60 40 2000 -0. 54 N 20 80 0 Prob>100 H= Correctly fcst El Ninos Observed El Ninos H =2/5=0. 4 F= Wrongly fcst El Ninos Non Obs El Ninos F =0/15=0. 0

ROC curve and area ROC curve 1, 00 El Niño Warning H F 0, 80 0, 70 ≥ 100% 0, 60 ≥ 60% ≥ 40% Hit rate Never ≥ 80% Neutral 0, 90 H La Niña F H F 0, 50 0, 40 0, 30 no skill 0, 20 ≥ 20% 0, 10 ≥ 0% 0, 00 0 0, 2 0, 4 0, 6 False-alarm rate 0, 8 1

ROC curve and area ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H La Niña F H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 0, 50 0, 40 ≥ 60% 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 0, 10 ≥ 0% 1. 00 0, 30 no skill 0, 20 0 0, 2 0, 4 0, 6 False-alarm rate 0, 8 1

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H La Niña F H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 0, 50 0, 40 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H La Niña F (0. 8, 0. 33) H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 0, 50 0, 40 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H (0. 8, 0. 2) La Niña F (0. 8, 0. 33) H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 0, 50 0, 40 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H (0. 8, 0. 2) La Niña F (0. 8, 0. 33) H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 (0. 6, 0. 0) 0, 50 0, 40 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H (0. 8, 0. 2) La Niña F (0. 8, 0. 33) H F 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 0, 60 (0. 6, 0. 0) 0, 50 0, 40 0. 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 0. 60 (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 0. 60 (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 0. 60 (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

Trapezoid area A = 0. 5*(a+b)*h a A h b

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 ≥ 60% 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 0. 60 (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 A 1 = 0. 5*(1. 0+0. 8)*0. 67 = 0. 603 0, 20 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 A 2 = 0. 5*(0. 8+0. 8)*0. 13 = 0. 104 0, 20 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 A 3 = 0. 5*(0. 6+0. 8)*0. 2 = 0. 140 0, 20 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 A = 0. 603 + 0. 104 + 0. 140 = 0. 847 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

ROC curve and area (1. 0, 1. 0) ROC curve 1, 00 El Niño H F Never 0. 00 ≥ 100% 0. 40 0. 00 0. 60 0. 00 0, 80 H 0, 60 0. 00 ≥ 40% 0. 80 0. 20 ≥ 20% 0. 80 0. 33 ≥ 0% 1. 00 F (0. 8, 0. 33) H F (0. 6, 0. 0) 0, 50 0, 40 ≥ 60% (0. 8, 0. 2) La Niña 0, 70 Hit rate Warning ≥ 80% Neutral 0, 90 (0. 4, 0. 0) 0, 30 E = 0. 8467 0, 20 no skill 0, 10 0, 00 (0. 0, 0. 0) 0, 20 0, 40 0, 60 False-alarm rate 0, 80 1, 00

Exercises • Plot the ROC curves and compute ROC area for Neutral or La Nina

ROC curves – curve shapes

ROC curves – curve shape (1) below-normal rainfall category (dry / non-dry years) 1 The forecasts are perfectly by discriminating dry years from non-dry years. 0, 9 0, 8 0, 7 Hit Rate 0, 6 The forecast probabilities for all the dry years are higher than the forecast probabilities for all the non-dry years. 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1

ROC curves – curve shape (2) below-normal rainfall category (dry / non-dry years) 1 The forecasts are no better by discriminating dry years from non-dry years than by guessing. 0, 9 0, 8 0, 7 Hit Rate 0, 6 There are just two possibilities, guessing has 50% chance of discriminating correctly. 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1

ROC curves – curve shape (3) below-normal rainfall category (dry / non-dry years) 1 The forecasts are very well at discriminating dry years from non-dry years. (ROC area is about 95%) 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1

ROC curves – curve shape (4) below-normal rainfall category (dry / non-dry years) 1 The forecasts are good by discriminating dry years from non-dry years. (ROC area about 75%) 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1

ROC curves – curve shape (5) below-normal rainfall category (dry / non-dry years) 1 The forecasts can discriminate dry years from non-dry years , but most of the time they indicate the incorrect year as dry. 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1

ROC curves – curve shape (6) below-normal rainfall category (dry / non-dry years) 1 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1 • The forecasts are fairly by discriminating dry years from non-dry years, better than just by guessing (ROC area about 60%). • The forecasts are performing well at identifying dry year occurs. The forecast probabilities are high when a dry year occurs. • The forecasts are performing poorly at identifying non-dry years.

ROC curves – curve shape (7) below-normal rainfall category (dry / non-dry years) 1 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1 • The forecasts are fairly by discriminating dry years from non-dry years, better than just by guessing (ROC area about 60%). • The forecasts are performing well at identifying non-dry years. The forecast probabilities are low when a dry year does not occur. • The forecasts are performing poorly at identifying dry years.

ROC curves – curve shape (8) below-normal rainfall category (dry / non-dry years) 1 0, 9 0, 8 0, 7 Hit Rate 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 False-alarm Rate 0, 8 0, 9 1 • The forecasts are fairly by discriminating dry years from nondry years, better than just by guessing (ROC area about 60%). • The forecasts are performing well at identifying dry years. The forecast probabilities are high when a dry year occurs. • The forecasts are performing well at identifying non-dry years. The forecast probabilities are low when a dry year does not occur. • The forecasts are performing poorly when the probabilities are close to climatology.