FSUTMSCube Calibration Standards Draft Guidelines and Standards presented

FSUTMS-Cube Calibration Standards Draft Guidelines and Standards presented to Florida Model Task Force Model Advancement Committee presented by Robert G. Schiffer, AICP Thomas F. Rossi Cambridge Systematics, Inc. Yongqiang Wu, P. E. Florida Department of Transportation November 28, 2007 Transportation leadership you can trust.

Presentation Overview Literature Review Default Guidelines and Standards (LRTP transit models) • Checking Input Data • Trip Generation • Trip Distribution • Mode Choice • Trip Assignment Validation for Other Model Applications Discussion on Volume-over-Count Ratios Next Steps 1

Project Overview Follow-up to Phase I Study on model parameters • Prepared summary of NHTS statistics for potential model use • Identified adjustable parameters and potential sources for defaults • Final report is available at the FSUTMSOnline web site − http: //www. fsutmsonline. net/images/uploads/mtf-files/FSUTMSCube_Parameters. pdf Phase II Study on calibration standards includes 4 subtasks • • Literature Review (complete) Model Calibration/Validation Guidelines and Standards (today) Best Practices for Model Calibration/Validation (next) Documentation – 1) Calibration and Validation Standards; 2) Best Practices for Model Validation; 3) Guidelines for Model Application 2

Project Overview (continued) Calibration vs. validation • Calibration – process where models are adjusted to simulate or match observed travel behavior in the study area • Validation – procedure used to adjust models to simulate base-year traffic counts and transit ridership figures Standards vs. guidelines/benchmarks • Standards – desirable accuracy levels for comparing estimated versus observed metrics • Benchmarks – documented statistical ranges from literature review, model outputs, NHTS, etc. Literature review • 60+ documents reviewed – specific models and reference reports 3

Default Guidelines and Standards Checking Input Data Socioeconomic data • Visual comparisons • Statistical comparisons • Regionwide comparisons (below) County − Persons per DU (or HH) − Employment/ population ratio − Autos/DU (or HH) Census Data 2000 2003 Duval 779, 618 817, 480 Clay 141, 671 Percent Difference 2005 5% 762, 674 810, 493 6% 157, 502 11% 139, 036 167, 020 20% 57, 903 61, 625 6% 56, 897 64, 695 14% 124, 458 142, 869 15% 120, 738 150, 084 24% 1, 103, 650 1, 179, 476 7% 1, 079, 345 1, 192, 292 10% St Johns Benchmarks/Settings Low High Regionwide Persons/DU (or HH) 2. 0 2. 7 Regionwide Employment/Population Ratio 0. 45 0. 75 Regionwide Autos/DU (or HH) 1. 75 2. 10 Approximate Population/TAZ N/A 4 3, 000 Statistic Percent Difference 2000 Nassau Total NERPM Data

Default Guidelines and Standards Checking Input Data (continued) Highway network data Transit network data Highway and transit speed data • Hierarchy • Balance highway and transit Terminal times • Hierarchy • Phase I Report 5

Default Guidelines and Standards Trip Generation Benchmarksa Aggregate trip rates • Person trips/TAZ • Person trips/person • Person trips/DU (or HH) a Statistic Low High Person Trips/TAZ N/A 15, 000 Person Trips/Person 3. 3 4. 0 Person Trips/DU (or HH) 8. 0 10. 0 HBW Person Trips/Employee 1. 20 1. 55 Generally excludes nonmotorized trips; including motorized trips could increase person trips per DU up to 11. 5. • HBW person trips/employee Total unbalanced attractions versus productions by purpose • Preferred +/-10%; acceptable in some instances +/-50% Percent external-external trips by zone/station • Great variation expected (4 -21 percent range documented) 6

Default Guidelines and Standards Trip Generation (continued) Percent trips by purpose Benchmarks Statistic Low (Percent) High (Percent) Percent Trips by Purpose – HBW 12 24 Percent Trips by Purpose – HBSH 10 20 Percent Trips by Purpose – HBSR 9 12 Percent Trips by Purpose – HBSC 5 8 Percent Trips by Purpose – HBOa 14 28 Percent Trips by Purpose – HBNWb 45 60 Percent Trips by Purpose – NHBc 20 33 a HBO includes a variety of special trip purposes depending on the model (e. g. , airport, college, and shop). b HBNW accounts for all home-based trip purposes except HBW. c NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate. 7

Default Guidelines and Standards Trip Distribution Benchmarks Average trip length by purpose Trip length frequency distributions by purpose Coincidence ratios by purpose – measures the percent of area that coincides for two trip length frequencies 12 35 Average Trip Length – HBSH (minutes) 9 19 Average Trip Length – HBSR (minutes) 11 19 Average Trip Length – HBSC (minutes) 7 16 (minutes) 8 20 Average Trip Length – NHBb (minutes) 6 19 Average Trip Length – IE (minutes) 26 58 HBOa Estimated (ATL = 18. 2 Min) 6% +/-3% Trip Length Frequency Distribution versus Observed +/-5% 2% 20 30 Travel Time (in Minutes) 40 50 8 60 70% a HBO includes a variety of special trip purposes, depending on the model (e. g. , airport, college, and school). b NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate. c Some lower coincidence ratios have been deemed acceptable for trip purposes that had relatively few trips and therefore higher error rates. Observed (ATL = 18. 9 Min) 4% Standards Mean Trip Length, Observed Total Trips Coincidence Ratios by Purposec Coincidence Ratio = 0. 82 10 Average Trip Length – HBW (minutes) Statistic 8% 0 High Average Trip Length – Percent of Total Trips 0% Low Statistic

Default Guidelines and Standards Trip Distribution (continued) Percent intrazonal trips by purpose Node-point charts • Zone-based • Number of trips • Trip productions/attractions by purpose Benchmarks Statistic Low High Percent Intrazonal – HBW 1% 4% Percent Intrazonal – HBSH 3% 9% Percent Intrazonal – HBSR 4% 10% Percent Intrazonal – HBSC 10% 12% Percent Intrazonal – HBOa 3% 7% a Percent Intrazonal – NHBb 5% 9% b Percent Intrazonal – Total Trips 3% 5% Standards Statistic Percent Intrazonal, Observed Total Trips Acceptable +/-3% Preferable 9+/-5% HBO includes a variety of special trip purposes, depending on the model (e. g. , airport, college, and school). NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

Default Guidelines and Standards Mode Choice Mode split targets (ideal) • • Trip purpose Mode Auto ownership level Geographic subarea Mode Zero-Vehicle Households One-Vehicle Households Two-Vehicle Households Three-Vehicle Households Walk 5, 000 6, 000 4, 000 3, 000 Bike 2, 000 1, 000 500 200 - 130, 000 350, 000 200, 000 Shared Ride 2 Persons 6, 000 15, 000 20, 000 10, 000 Shared Ride 3 Persons 1, 000 2, 000 4, 000 2, 000 Local Bus, Walk 6, 000 7, 000 4, 000 1, 000 Local Bus, PNR - 500 Local Bus, KNR - 200 2, 000 500 Express Bus, Walk 1, 000 500 Express Bus, PNR - 2, 000 4, 000 2, 000 Express Bus, KNR - 200 500 1, 000 400 - 300 500 Drive Alone LRT, Walk LRT, PNR LRT, KNR 10

Default Guidelines and Standards Mode Choice (continued) Mode splits by observed calibration targets Total area transit trips, estimated versus observed Transit trips between districts • Tabular comparisons (CTPP) • Desire lines Mean trip length, estimated transit trips versus observed Standards Statistic Total Area Transit Trips versus Observed Transit Trips between Districts Low High +/-1% +/- 2% Compare model trip table against CTPP or HH survey Mean Trip Length Transit Trips versus Observed +/-5% +/-15% Mode Splits by Observed Calibration Targets +/- 2% -0. 6 -0. 1 Elasticity of Demand with Respect to LOS Variables 11

Default Guidelines and Standards Trip Assignment Standards Acceptable Preferable Freeway Volume-over-Count +/- 7% +/- 6% Arterial Volume-over-Count +/- 15% +/- 10% Collector Volume-over-Count +/- 25% +/- 20% Frontage Road Volume-over-Count +/- 25% Statistic Volume-over-count ratios +/-1 lane percent error (recalculated per FDOT LOS Handbook) Freeway Peak Volume-over-Count 75% of links @ +/-20%; 50% of links @ +/-10% Major Arterial Peak Volume-over-Count 75% of links @ +/-30%; 50% of links @ +/-15% Assigned VMT-over-Count Areawide +/-5% +/-2% Assigned VHT-over-Count Areawide +/-5% +/-2% Assigned VMT-over-Count by FT/AT/NL +/- 25% +/- 15% Assigned VHT-over-Count by FT/AT/NL +/- 25% +/- 15% Aggregate VMT Standards Acceptable Preferable Percent Error – LT 10, 000 volume (2 L road) 50% 25% Percent Error – 10, 000 -30, 000 (4 L road) 30% 20% Percent Error – 30, 000 -50, 000 (6 L road) 25% 15% Percent Error – 50, 000 -65, 000 (4 -6 L freeway) 20% 10% Percent Error – 65, 000 -75, 000 (6 L freeway) 15% 5% Percent Error – GT 75, 000 (8+L freeway) 10% 5% Statistic • VMT/HH (60 -75) • VMT/person (24 -32) • VMT/commercial vehicle (3 -25%) 12

Default Guidelines and Standards Trip Assignment (continued) Screenline volume-over-count RMSE by volume group Transit assignment validation Benchmarks Statistic Estimated-over-Observed Transit Trips Low High +/- 9% +/- 3% Standards Statistic Acceptable Preferable Acceptable Error – Transit Screenlines +/-20% +/-10% Transit Ridership – <1, 000 Passengers/Day +/-150% +/- 100% Transit Ridership – 1 k-2 k Passengers/Day +/- 100% +/- 65% Transit Ridership – 2 k-5 k Passengers/Day +/- 65% +/- 35% Transit Ridership – 5 k-10 k Passengers/Day +/- 35% +/- 25% Transit Ridership – 10 k-20 k Passengers/Day +/- 25% +/- 20% Transit Ridership – >20, 000 Passengers/Day +/- 20% +/- 15% 13 Standards Acceptable Preferable RMSE – LT 5, 000 AADT 150% 45% RMSE – 5, 000 -9, 999 AADT 45% 35% RMSE – 10, 000 -14, 999 AADT 35% 27% RMSE – 15, 000 -19, 999 AADT 35% 25% RMSE – 20, 000 -29, 999 AADT 27% 15% RMSE – 30, 000 -49, 999 AADT 25% 15% RMSE – 50, 000 -59, 999 AADT 20% 10% RMSE – 60, 000+ AADT 19% 10% RMSE Areawide 45% 35% Statistic

Other Model Applications LRTP Highway Only Models Same default guidelines and standards except Benchmarks/Settings • Replace mode choice checks with auto occupancy comparisons against NHTS and other surveys Statistic Low High Auto Occupancy Rates – HBW 1. 05 1. 10 Auto Occupancy Rates – HBSH 1. 50 1. 80 Auto Occupancy Rates – HBSR 1. 70 1. 90 Auto Occupancy Rates – HBOa 1. 65 1. 95 NHBb 1. 60 1. 90 Auto Occupancy Rates – Auto Occupancy Rates Current Model 1988 FLSWM 2001 NHTS FL 2001 NHTS US HBW 1. 10 1. 30 1. 06 1. 10 HBShop 1. 80 1. 55 1. 57 1. 80 HBSR 1. 94 2. 27 1. 79 1. 94 HBO 1. 70 1. 50 1. 90 1. 70 NHB 1. 71 1. 58 1. 82 1. 71 Purpose • Commercial vehicle VMT checks not likely relevant • No transit assignment validation 14 a HBO includes a variety of special trip purposes, depending on the model (e. g. , airport, college, and school). b NHB includes combined purposes for NHB Work and NHB Non-Work, where appropriate.

Other Model Applications FTA New Starts Models Transit networks and pathbuilding checks Acceptable Range of Values • Compare skim settings to on-board surveys Statistic Low Elasticity of demand with respect to LOS variables -0. 10 IVT parameter – HBW* -0. 02 -0. 03 0. 1 to 0. 5*CIVT HBW trips ~CIVT HBW trips IVT parameter – HBNW* Trip distribution checks IVT parameter – NHB* Mode choice calibration Highway assignment checks Transit assignment checks Ratio – OVT/IVT parameters – HBW* 2. 0 Ratio – OVT/IVT parameters – HBNW* 2. 0 Ratio – OVT/IVT parameters – NHB* 2. 0 Implied value of time – Percent of income 25% • Assign on-board survey trip table and compare ridership Implied value of time – HBW $2. 00 Implied value of time – HBNW $0. 50 • SUMMIT diagnostics Implied value of time – NHB $0. 20 * FTA published guideline. 15 High -0. 70 3. 0 33% $7. 00 $5. 00

Other Model Applications Subarea Models Prerequisite – approved regional model validation Input data – focus on socioeconomic and network data Trip generation – review and compare subarea versus regional model aggregate trip rates Trip distribution – compare subarea versus regional average trip length and percent intrazonal trips by purpose Mode choice – check subarea mode shares versus regional Trip assignment – volume-over-count (v-o-c), percent error, VMT and VHT v-o-c, v-o-c by screenline/cutline, and RMSE 16

Other Model Applications Corridor Models Same subarea model validation checks Input data – focus on network details surrounding corridor Trip generation – review corridor productions and attractions by zone Trip distribution – desire line analyses Mode choice – review of mode shares within study area Trip assignment – more stringent standards for v-o-c, vo-c by screenline/cutline 17 Standards Acceptable Preferable Freeway Volume-over-Count +/- 6% +/- 5% Arterial Volume-over-Count +/- 10% +/- 7% Collector Volume-over-Count +/- 15% +/- 10% Frontage Rd Volume-over-Count +/- 20% +/- 15% Statistic

Other Model Applications Models for DRIs and Other Impact Studies Input data • SE data – site, nearby zone assumptions, pop/TAZ • Networks – verify coding, path traces from site • Transit – access coding, headways, stop locations near site Trip generation – document trip rate assumptions Trip distribution – district summaries Mode choice – check ITE trips versus model trips Trip assignment – select zone and select link, turn volumes 18

An Issue for the MTF to Discuss… Discussion on volume-over-count ratios Ratio of summed modeled volumes for group of links and the sum of count volumes on the links (should be near 1. 0) This check does appear in the draft report at this time In the opinion of some, this is mathematically erroneous because of double counting It is somewhat duplicative of VMT checks 19

An Example V = 7, 500 C = 5, 000 v/c = (7, 500+5, 000)/ (5, 000+5, 000) V = 5, 000 = 1. 33 C = 5, 000 1 mile 20

An Example (continued) V = 5, 000 C = 5, 000 v/c = (5, 000+7, 500)/ (5, 000+5, 000) V = 7, 500 = 1. 17 C = 5, 000 1 mile 21

Proposed Solution Use VMT check • Example has same solution for both cases VMT(m) / VMT(c) = 1. 25 VMT is not double counted Screenline/cutline checks should be retained since double counting should not be an issue 22

Next Steps Take comments from MTF committee today Revise draft guidelines and standards based on FDOT and MTF committee comments Prepare technical report on best practices Develop guidelines for model application work 23