Many of the conclusions presented here are specific to the WFRC model and our ActivitySim implementation. However, some conclusions can apply more broadly….
6.1 Computational resources
As part of our recommendations, we include a discussion of the computational resources used for each model.
All runs of the WFRC model were done on a Windows 10 computer with 2 Intel Xeon Silver 4114 CPUs. The CPUs have a base frequency of 2.2 GHz with a maximum turbo frequency of 3.0 GHz, and 10 cores/20 threads each. The WFRC model is configured for multiprocessing in its destination and mode choice steps, and was configured to use 16 threads for our scenario runs. This machine also has 128 GB of RAM installed, and we need to check how much it actually uses. There were not significant differences in runtimes between each model scenario, and each scenario had a runtime of 16–17 hours.
Most runs of ActivitySim were done on nodes of the BYU supercomputer. Each node runs Red Hat Enterprise Linux 7.9, and uses an AMD EPYC 7763 CPU at 2.45 GHz. Each ActivitySim run requested 12 CPU cores and 360 GB of RAM. Running in single-threaded mode (i.e. only one CPU core was utilized), each run took roughly 5 hours to complete, and used nearly all of the 360 GB of RAM available. With multi-threading enabled, however, the runtimes decreased to around an hour per scenario, using 72% of the available CPU time across all 12 cores and 88% of the available RAM.
This is a huge difference in runtime between the two models, though crucially ActivitySim had 3 times as much RAM available for use. ActivitySim offers “chunking” options (asim-chunking?), where large tables are loaded into RAM in chunks rather than all at once. This can dramatically reduce the amount of RAM required to run an ActivitySim scenario, at the expense of increased runtimes. For comparison, we ran the baseline scenario in ActivitySim on the same computer used for the WFRC model scenarios, with chunking enabled to account for the amount of RAM available. With multi-threading set to use 36 of the 40 available (should probably match what the WFRC model actually uses if not the max amount threads, the baseline ActivitySim scenario ran in XXX hours (still need to do this).
6.2 Time Spent
In order to change from a trip-based to an activity-based model, agencies will need to spend time learning….
Table 6.1 shows the amount of time spent on creating and analyzing each scenario in both models. These are rough approximations, as detailed time logs are not available, but should serve to give a general idea of the time required to learn activitysim. Note as well that these tables show time spent by one graduate and one undergraduate research assistant. More experienced modelers would likely require significantly less time to create and analyze similar scenarios.
Table 6.1: time spent
mpg
cyl
disp
hp
drat
wt
qsec
vs
am
gear
carb
21.0
6
160
110
3.90
2.620
16.46
0
1
4
4
21.0
6
160
110
3.90
2.875
17.02
0
1
4
4
22.8
4
108
93
3.85
2.320
18.61
1
1
4
1
21.4
6
258
110
3.08
3.215
19.44
1
0
3
1
18.7
8
360
175
3.15
3.440
17.02
0
0
3
2
18.1
6
225
105
2.76
3.460
20.22
1
0
3
1
task
hours
Synthetic population creation
50
Baseline mode choice calibration
20
Add remote work models to ActivitySim
20
Baseline remote work calibration
10
Scenario creation: Land Use
15
Scenario creation: Transit
2
Scenario creation: Remote Work
5
6.3 Notes
Each model works differently. - There will be a learning curve when changing from TBM to ABM - Planning agencies should take this into account when switching
ABM is more malleable. It was easy to throw on the WFH model without having to completely change the rest of the model, and it seemed more realistic than the TBM. - For planning agencies wanting a model that more easily adapts to unforeseen travel behavior change, an ABM would be preferable
More analyses can be done with ABMs - We were able to replicate each TBM analysis with the ABM and more - We could make more demographic-type analyses with ABM - We were able to compare changes in individual behavior with ABM - Simpler to go about the analysis when thinking on the individual level
ActivitySim doesn’t have the built in analysis tools like CUBE does (I think this is true, but not sure) - We had to use different programs to analyze ActivitySim output (like R)