Config File

E4ST.read_config — Function

read_config(filename; create_out_path = true, kwargs...) -> config::OrderedDict{Symbol,Any}

read_config(filenames; create_out_path = true, kwargs...) -> config::OrderedDict{Symbol,Any}

read_config(path; create_out_path = true, kwargs...) -> config::OrderedDict{Symbol, Any}

Load the config file from filename, inferring any necessary settings as needed. If path given, checks for joinpath(path, "config.yml"). This can be used with the out_path returned by run_e4st See read_data to see how the config is used. If multiple filenames given, (in a vector, or separated by commas) merges them, preserving the settings found in the last file, when there are conflicts, appending the list of Modifications. Uses summarize_config to infer defaults, when applicable. Any specified kwargs are added to the config, over-writing anything except the list of Modifications.

The Config File is a file that fully specifies all the necessary information. Note that when filenames are given as a relative path, they are assumed to be relative to the location of the config file.

name	required	default	description
`base_out_path`	true	nothing	The path (relative or absolute) to the desired output folder. This folder doesn't necessarily need to exist. The code will make it for you if it doesn't exist yet. E4ST will make a timestamped folder within `base_out_path`, and store that new path into `config[out_path]`. This is to prevent processes from overwriting one another.
`gen_file`	true	nothing	The filepath (relative or absolute) to the generator table. See `summarize_table(::Val{:gen})`.
`bus_file`	true	nothing	The filepath (relative or absolute) to the bus table. See `summarize_table(::Val{:bus})`.
`branch_file`	true	nothing	The filepath (relative or absolute) to the branch table. See `summarize_table(::Val{:branch})`.
`hours_file`	true	nothing	The filepath (relative or absolute) to the hours table. See `summarize_table(::Val{:hours})`.
`nominal_load_file`	true	nothing	The filepath (relative or absolute) to the time representation. See `summarize_table(::Val{:nominal_load})`
`years`	true	nothing	a list of years to run in the simulation specified as a string. I.e. `"y2030"`
`optimizer`	true	nothing	The optimizer type and attributes to use in solving the linear program. The `type` field should be always be given, (i.e. `type: HiGHS`) as well as each of the solver options you wish to set. E4ST is a BYOS (Bring Your Own Solver :smile:) library, with default attributes for HiGHS and Gurobi. For all other solvers, you're on your own to provide a reasonable set of attributes. To see a full list of solvers with work with JuMP.jl, see here.
`mods`	false	OrderedCollections.OrderedDict{Symbol, E4ST.Modification}()	A list of `Modification`s specifying changes for how E4ST runs. See the `Modification` for information on what they are, how to add them to a config file.
`year_gen_data`	true	nothing	The year string (i.e. `y2016`) corresponding to the data year of the generator table.
`log_model_summary`	false	false	Whether or not to log a numerical summary of the model. Useful for debugging, but can take a while if the model is large.
`out_path`	false	nothing	the path to output to. If this is not provided, an output path will be created `make_out_path!`.
`other_config_files`	false	nothing	A list of other config files to read. Note that the options in the parent file will be honored.
`af_file`	false	nothing	The filepath (relative or absolute) to the availability factor table. See `summarize_table(::Val{:af_table})`
`cf_threshold`	false	0.001	The threshold below which the maximum capacity factor is considered to be zero. This helps with numerical performance of the solver. For example, a solar unit with an hourly average CF of 0.00001 will not operate, with the default `cf_threshold` of 1e-3.
`iter`	false	E4ST.RunOnce()	The `Iterable` object to specify the way the sim should iterate. If nothing specified, defaults to run a single time via `RunOnce`. Specify the `Iterable` type, and all keyword arguments.
`load_shape_file`	false	nothing	a file for specifying the hourly shape of load elements. See `summarize_table(::Val{:load_shape})`
`load_match_file`	false	nothing	a file for specifying annual load energy to match for sets. See `summarize_table(::Val{:load_match})`
`load_add_file`	false	nothing	a file for specifying additional load energy, after matching. See `summarize_table(::Val{:load_add})`
`load_add_file`	false	nothing	a file for specifying additional load energy, after matching. See `summarize_table(::Val{:load_add})`
`build_gen_file`	false	nothing	a file for specifying generators that could get built. See `summarize_table(::Val{:build_gen})`
`gentype_genfuel_file`	false	nothing	a file for storing gentype-genfuel pairings. See `summarize_table(::Val{:genfuel})`
`summary_table_file`	false	nothing	a file for giving information about additional columns not specified in `summarize_table`
`save_data`	false	true	A boolean specifying whether or not to save the loaded data to file for later use (i.e. by specifying a `data_file` for future simulations).
`data_file`	false	nothing	The filepath (relative or absolute) to the data file (a serialized julia object). If this is provided, it will use this instead of loading data from all the other files.
`results_formulas_file`	false	nothing	The filepath (relative or absolute) to the results formulas file. See `summarize_table(::Val{:results_formulas})`
`save_model_presolve`	false	false	A boolean specifying whether or not to save the model before solving it, for later use (i.e. by specifying a `model_presolve_file` for future sims). Defaults to `false`
`model_presolve_file`	false	nothing	The filepath (relative or absolute) to the unsolved model. If this is provided, it will use this instead of creating a new model.
`save_data_parsed`	false	true	A boolean specifying whether or not to save the raw results after solving the model. This could be useful for calling `process_results!(config)` in the future. Defaults to `true`
`save_data_processed`	false	true	A boolean specifying whether or not to save the processed results after solving the model. Defaults to `true`.
`objective_scalar`	false	1000.0	This is specifies how much to scale the objective by for the sake of the solver. Does not impact any user-created expressions or shadow prices from the raw results, as they get scaled back. (Defaults to 1e6)
`pgen_scalar`	false	1000.0	This specifies how much to scale pgen by in the conspgenmax constraint. Helps with numerical stability if there are small availability factors present. See also cf_threshold
`pcap_retirement_threshold`	false	1.0e-6	This is the minimum `pcap` threshold (in MW) for new generators to be kept. Defaults to 1e-6 (i.e. 1W). See also `save_updated_gen_table`
`voll`	false	5000	This is the assumed value of lost load for which the objective function will be penalized for every MWh of curtailed load.
`logging`	false	true	This specifies whether or not E4ST will log to `get_out_path(config, "E4ST.log")`. Options include `true`, `false`, or `"debug"`. See `start_logging!` for more info.
`eor_leakage_rate`	false	0.5	The assumed rate (between 0 and 1) at which CO₂ stored in Enhanced Oil Recovery (EOR) leaks back into the atmosphere.
`line_loss_rate`	false	0.1	The assumed electrical loss rate from generation to consumption, given as a ratio between 0 and 1. Default is 0.1, or 10% energy loss
`line_loss_type`	false	plserv	The term in the power balancing equation that gets penalized with line losses. Can be "pflow" or "plserv". Using "pflow" is more accurate in that it accounts for only losses on power coming from somewhere else, at the expense of a larger problem size and greater solve time. Default is `plserv` due to increased runtime with `pflow`
`distribution_cost`	false	60	The assumed cost per MWh of served power, for the transmission and distribution of the power.
`bio_pctco2e`	false	0.273783186	The fraction of biomass co2 emissions that are considered new to the atmostphere. 0.225 metric tons/MWh * (2204 short tons/2000 metric tons) / 0.904 short tons/MWh
`ng_upstream_ch4_leakage`	false	0.000434	Natural gas methane fuel content. (Short ton/MMBtu)
`coal_upstream_ch4_leakage`	false	0.000175	Coal methane fuel content. (Short ton/MMBtu)
`wacc`	false	0.0544	Assumed Weighted Average Cost of Capital (used as discount rate), currently only used for calculating ptc capex adjustment but should be the same as the wacc/discount rate used to calculate annualized generator costs. Current value (0.0544) was using in annulaizing ATB 2022 costs.
`error_if_zero_af`	false	true	Whether or not to throw an error if there are generators with zero availability over the entire year. If set to equal false, it will throw a warning message rather than an error.

Example Config File

base_out_path:    "../out/3bus"
gen_file:    "../data/3bus/gen.csv"
bus_file:    "../data/3bus/bus.csv"
branch_file: "../data/3bus/branch.csv"
hours_file:   "../data/3bus/hours.csv"
af_file:     "../data/3bus/af.csv"
nominal_load_file:     "../data/3bus/load.csv"
gentype_genfuel_file: "../data/3bus/gentype_genfuel.csv"
build_gen_file: "../data/3bus/build_gen.csv"
test_param1: "RIGHT" # This should not get overwritten by config_other
year_gen_data: y2020
other_config_files:
  - "config_other.yml"
years:
  - y2030
  - y2035
  - y2040
optimizer:
  type: "HiGHS" # See available options here: https://docs.juliahub.com/HiGHS/WFic5/0.3.1/
  dual_feasibility_tolerance: 1e-5
  log_file: HiGHS.log
  log_to_console: false
  solver: simplex # "ipm" is probably preferred for larger problems, but it logs to console due to an issue
mods:
  example_policy:
    type: "ExamplePolicyType"
    value: 0
    some_parameter:
      - "This makes an "
      - "array of strings."
      - "Cool, right?"
    other_parameter:
      name: "This makes an OrderedDict"
  other_mod:
    type: "OtherModificationType"
    value: 0
    custom_parameter: "hello!!!"

source

E4ST.summarize_config — Function

summarize_config() -> summary::DataFrame

Summarizes the config, with columns for:

name - the property name, i.e. key
required - whether or not the property is required
default - default value of this property
description

name	required	default	description
`base_out_path`	true	nothing	The path (relative or absolute) to the desired output folder. This folder doesn't necessarily need to exist. The code will make it for you if it doesn't exist yet. E4ST will make a timestamped folder within `base_out_path`, and store that new path into `config[out_path]`. This is to prevent processes from overwriting one another.
`gen_file`	true	nothing	The filepath (relative or absolute) to the generator table. See `summarize_table(::Val{:gen})`.
`bus_file`	true	nothing	The filepath (relative or absolute) to the bus table. See `summarize_table(::Val{:bus})`.
`branch_file`	true	nothing	The filepath (relative or absolute) to the branch table. See `summarize_table(::Val{:branch})`.
`hours_file`	true	nothing	The filepath (relative or absolute) to the hours table. See `summarize_table(::Val{:hours})`.
`nominal_load_file`	true	nothing	The filepath (relative or absolute) to the time representation. See `summarize_table(::Val{:nominal_load})`
`years`	true	nothing	a list of years to run in the simulation specified as a string. I.e. `"y2030"`
`optimizer`	true	nothing	The optimizer type and attributes to use in solving the linear program. The `type` field should be always be given, (i.e. `type: HiGHS`) as well as each of the solver options you wish to set. E4ST is a BYOS (Bring Your Own Solver :smile:) library, with default attributes for HiGHS and Gurobi. For all other solvers, you're on your own to provide a reasonable set of attributes. To see a full list of solvers with work with JuMP.jl, see here.
`mods`	false	OrderedCollections.OrderedDict{Symbol, E4ST.Modification}()	A list of `Modification`s specifying changes for how E4ST runs. See the `Modification` for information on what they are, how to add them to a config file.
`year_gen_data`	true	nothing	The year string (i.e. `y2016`) corresponding to the data year of the generator table.
`log_model_summary`	false	false	Whether or not to log a numerical summary of the model. Useful for debugging, but can take a while if the model is large.
`out_path`	false	nothing	the path to output to. If this is not provided, an output path will be created `make_out_path!`.
`other_config_files`	false	nothing	A list of other config files to read. Note that the options in the parent file will be honored.
`af_file`	false	nothing	The filepath (relative or absolute) to the availability factor table. See `summarize_table(::Val{:af_table})`
`cf_threshold`	false	0.001	The threshold below which the maximum capacity factor is considered to be zero. This helps with numerical performance of the solver. For example, a solar unit with an hourly average CF of 0.00001 will not operate, with the default `cf_threshold` of 1e-3.
`iter`	false	E4ST.RunOnce()	The `Iterable` object to specify the way the sim should iterate. If nothing specified, defaults to run a single time via `RunOnce`. Specify the `Iterable` type, and all keyword arguments.
`load_shape_file`	false	nothing	a file for specifying the hourly shape of load elements. See `summarize_table(::Val{:load_shape})`
`load_match_file`	false	nothing	a file for specifying annual load energy to match for sets. See `summarize_table(::Val{:load_match})`
`load_add_file`	false	nothing	a file for specifying additional load energy, after matching. See `summarize_table(::Val{:load_add})`
`load_add_file`	false	nothing	a file for specifying additional load energy, after matching. See `summarize_table(::Val{:load_add})`
`build_gen_file`	false	nothing	a file for specifying generators that could get built. See `summarize_table(::Val{:build_gen})`
`gentype_genfuel_file`	false	nothing	a file for storing gentype-genfuel pairings. See `summarize_table(::Val{:genfuel})`
`summary_table_file`	false	nothing	a file for giving information about additional columns not specified in `summarize_table`
`save_data`	false	true	A boolean specifying whether or not to save the loaded data to file for later use (i.e. by specifying a `data_file` for future simulations).
`data_file`	false	nothing	The filepath (relative or absolute) to the data file (a serialized julia object). If this is provided, it will use this instead of loading data from all the other files.
`results_formulas_file`	false	nothing	The filepath (relative or absolute) to the results formulas file. See `summarize_table(::Val{:results_formulas})`
`save_model_presolve`	false	false	A boolean specifying whether or not to save the model before solving it, for later use (i.e. by specifying a `model_presolve_file` for future sims). Defaults to `false`
`model_presolve_file`	false	nothing	The filepath (relative or absolute) to the unsolved model. If this is provided, it will use this instead of creating a new model.
`save_data_parsed`	false	true	A boolean specifying whether or not to save the raw results after solving the model. This could be useful for calling `process_results!(config)` in the future. Defaults to `true`
`save_data_processed`	false	true	A boolean specifying whether or not to save the processed results after solving the model. Defaults to `true`.
`objective_scalar`	false	1000.0	This is specifies how much to scale the objective by for the sake of the solver. Does not impact any user-created expressions or shadow prices from the raw results, as they get scaled back. (Defaults to 1e6)
`pgen_scalar`	false	1000.0	This specifies how much to scale pgen by in the conspgenmax constraint. Helps with numerical stability if there are small availability factors present. See also cf_threshold
`pcap_retirement_threshold`	false	1.0e-6	This is the minimum `pcap` threshold (in MW) for new generators to be kept. Defaults to 1e-6 (i.e. 1W). See also `save_updated_gen_table`
`voll`	false	5000	This is the assumed value of lost load for which the objective function will be penalized for every MWh of curtailed load.
`logging`	false	true	This specifies whether or not E4ST will log to `get_out_path(config, "E4ST.log")`. Options include `true`, `false`, or `"debug"`. See `start_logging!` for more info.
`eor_leakage_rate`	false	0.5	The assumed rate (between 0 and 1) at which CO₂ stored in Enhanced Oil Recovery (EOR) leaks back into the atmosphere.
`line_loss_rate`	false	0.1	The assumed electrical loss rate from generation to consumption, given as a ratio between 0 and 1. Default is 0.1, or 10% energy loss
`line_loss_type`	false	plserv	The term in the power balancing equation that gets penalized with line losses. Can be "pflow" or "plserv". Using "pflow" is more accurate in that it accounts for only losses on power coming from somewhere else, at the expense of a larger problem size and greater solve time. Default is `plserv` due to increased runtime with `pflow`
`distribution_cost`	false	60	The assumed cost per MWh of served power, for the transmission and distribution of the power.
`bio_pctco2e`	false	0.273783186	The fraction of biomass co2 emissions that are considered new to the atmostphere. 0.225 metric tons/MWh * (2204 short tons/2000 metric tons) / 0.904 short tons/MWh
`ng_upstream_ch4_leakage`	false	0.000434	Natural gas methane fuel content. (Short ton/MMBtu)
`coal_upstream_ch4_leakage`	false	0.000175	Coal methane fuel content. (Short ton/MMBtu)
`wacc`	false	0.0544	Assumed Weighted Average Cost of Capital (used as discount rate), currently only used for calculating ptc capex adjustment but should be the same as the wacc/discount rate used to calculate annualized generator costs. Current value (0.0544) was using in annulaizing ATB 2022 costs.
`error_if_zero_af`	false	true	Whether or not to throw an error if there are generators with zero availability over the entire year. If set to equal false, it will throw a warning message rather than an error.

source

E4ST.save_config — Function

save_config(config) -> nothing

saves the config to the output folder specified inside the config file

source

E4ST.check_config! — Function

check_config!(config)

Ensures that config has required fields listed in summarize_config

source

E4ST.summarize_table — Method

summarize_table(s::Symbol) -> summary::DataFrame

Returns a summary of the table s. Note that more information can be provided in the the summary_table, which contains a summary of all tables, including all information from summarize_table, plus additional columns specified.

source

E4ST.make_out_path! — Function

make_out_path!(config) -> nothing

If config[:out_path] provided, does nothing. Otherwise, makes sure config[:base_out_path] exists, making it as needed. Creates a new time-stamped folder via time_string, stores it into config[:out_path]. See get_out_path to create paths for output files.

source

E4ST.get_out_path — Function

get_out_path(config, filename) -> path

Returns joinpath(config[:out_path], filename)

source