Difference between revisions of "WRF Hindcast"
(→Platforms) |
|||
| (20 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | Agrineer's WRF Hindcast Project generates the data used by the [[Grow Degree Calculator]] and the [[Soil Moisture Estimator]] applications. WRF stands for Weather, Research, and Forecasting and is a program made available by [ | + | Agrineer's WRF Hindcast Project generates the input data used by the [[Grow Degree Calculator]] (GDC) and the [[Soil Moisture Estimator]] (SME) applications. WRF stands for [http://www2.mmm.ucar.edu/wrf/users Weather, Research, and Forecasting] and is a program made available by [https://www.mmm.ucar.edu/wrf-model-general UCAR/NCAR] and other research participants. |
WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past. | WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past. | ||
| Line 7: | Line 7: | ||
[[File:WRF_Sectors.png|300px|thumb|right|WCONUS sector grid, current operational sectors are shown in green.]] | [[File:WRF_Sectors.png|300px|thumb|right|WCONUS sector grid, current operational sectors are shown in green.]] | ||
| − | The area modeled by the WRF program is called a "domain", but since this implementation uses three domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is made up of 171x171 "pixels", 3.3km x 3.3km in size. Client | + | The area modeled by the WRF program is called a "domain", but since this implementation uses three nested domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is currently made up of 171x171 "pixels", at 3.3km x 3.3km in size. Client applications can access the pixel's simulated weather data by indicating latitude and longitude coordinates. |
| + | <!-- TODO: insert image example of nested domains here --> | ||
| − | The image to the right depicts the sectors defined, and | + | The image to the right depicts the sectors defined, and currently active (operational) sectors are shown in green. The defined set of sectors are designated WCONUS (Western CONtinental United States) with an appellation of grid location. Sector rows are given letters and columns are given numbers, with the origin on the bottom right. For example, the bottom right sector is WCONUS_A0 and the top left sector is WCONUS_E4. |
| − | + | ||
| − | == | + | == Implementation == |
| − | The WRF climate simulation program requires a sophisticated input stream and is usually executed on | + | The WRF climate simulation program requires a sophisticated input stream, consisting of daily data and configuration files, and is usually executed on a parallel computing platform. See [[#Platforms|Platforms]] below for Agrineer's implementation. Each sector's output data is calculated off-line and delivered to the web server on a daily basis. Users can expect a maximum of one day delay, with eastern sectors getting results in the morning and western sectors getting results later in the day. "Namelist" input files to WPS and WRF programs can be found in the sector data download [https://www.agrineer.org/downloads/sectors.php page]. |
| − | == | + | == Output == |
| + | Typical WRF output include hundreds of variable values amounting to about 4.5 GB (1.5 GB per domain) per sector daily. A portion of these are used to calculate the standard reference evaporation (ETo) values in the [[Soil Moisture Estimator|SME]] application. Since the daily [[ETo Calculation|ETo]] values are calculated off-line using WRF output, we could deliver to the server just the ETo and precipitation values for accumulation purposes. However, the [[Grow Degree Calculator|GDC]] application requires daily maximum and minimun temperatures and so we deliver these as well, in netCDF format. All three domains for each sector are available on a daily basis from [https://www.agrineer.org/downloads/sectors.php here]. Use the netCDF programs "ncdump" and "ncview" to inspect the headers and contents. | ||
| − | == | + | == Global Forecast System == |
| − | Agrineer's implementation of the WRF program uses the [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs | + | Agrineer's implementation of the WRF program uses the [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs Global Forecast System] (GFS) forcing files as input. These are files generated by [https://www.ncdc.noaa.gov NOAA's National Centers for Environmental Information] (NCEI),which are used to force data interpolations to defined values at certain times, projected into the future at six hour intervals. Once a projected time is reached, the forcing file is regenerated using real data instead of projected data and a new set of projected files are generated, and the process repeats itself. This project uses the re-analyzed files at six-hour intervals for historical evaluation. The resolution of the data is 2.5 x 2.5 degrees, and the gridded data is interpolated down to a 0.25 X 0.25 degree resolution to provide finer detail. The data is retrieved from [ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/ here] where the folders are date and interval dependent. To retrieve re-analyzed data for 2017/03/03 interval 00, for example, go to gfs.2017030300, then find gfs.t00z.pgrb2.0p25.f000. |
| − | == | + | == Platform == |
| + | Agrineer's WRF platform is Linux Mint as the operating system on an 8-core 64-bit AMD based computer, compiled under GNU gfortran with MPICH parallel implementation. The Python language is used to integrate all of the working parts, from retrieving input files to uploading results to the server. The dynamic solver model used in WRF is the ARW (Advanced Research WRF) core. | ||
| − | + | The computer platform described can calculate 10 sectors per day, with rest periods for maintenance. For redundancy, Agrineer operates two WRF platforms in separate long distance locations. | |
| − | + | ||
| + | == WRF Docker Container == | ||
| + | A Docker container package is provided which generates WRF data used in the GDC and SME tools. Third party data contribution for specific sectors can be done with this package and is suitable for compute servers and multi-core desktops alike. The container package can be downloaded [https://gitlab.com/agrineer/WRF_container here]. Registration with Agrineer.org is required for data contributions used in the GDC and SME tools. | ||
| + | |||
| + | <!-- | ||
== FAQ == | == FAQ == | ||
| Line 30: | Line 35: | ||
=== Download and Install WRF === | === Download and Install WRF === | ||
=== Create Sector directories === | === Create Sector directories === | ||
| − | |||
| − | |||
| − | === | + | == Input Files == |
| + | === GFS === | ||
| + | === WPS and WRF namelist === | ||
| + | == Scripts == | ||
== User Support == | == User Support == | ||
| + | --> | ||
Latest revision as of 20:28, 18 January 2019
Agrineer's WRF Hindcast Project generates the input data used by the Grow Degree Calculator (GDC) and the Soil Moisture Estimator (SME) applications. WRF stands for Weather, Research, and Forecasting and is a program made available by UCAR/NCAR and other research participants.
WRF is normally used to forecast weather, but for our purposes we use it to simulate weather in the past.
Contents
Sectors
The area modeled by the WRF program is called a "domain", but since this implementation uses three nested domains (at 30km, 10km, and 3.3km resolution), the area of interest, the third domain, is called a "sector". Each sector is currently made up of 171x171 "pixels", at 3.3km x 3.3km in size. Client applications can access the pixel's simulated weather data by indicating latitude and longitude coordinates.
The image to the right depicts the sectors defined, and currently active (operational) sectors are shown in green. The defined set of sectors are designated WCONUS (Western CONtinental United States) with an appellation of grid location. Sector rows are given letters and columns are given numbers, with the origin on the bottom right. For example, the bottom right sector is WCONUS_A0 and the top left sector is WCONUS_E4.
Implementation
The WRF climate simulation program requires a sophisticated input stream, consisting of daily data and configuration files, and is usually executed on a parallel computing platform. See Platforms below for Agrineer's implementation. Each sector's output data is calculated off-line and delivered to the web server on a daily basis. Users can expect a maximum of one day delay, with eastern sectors getting results in the morning and western sectors getting results later in the day. "Namelist" input files to WPS and WRF programs can be found in the sector data download page.
Output
Typical WRF output include hundreds of variable values amounting to about 4.5 GB (1.5 GB per domain) per sector daily. A portion of these are used to calculate the standard reference evaporation (ETo) values in the SME application. Since the daily ETo values are calculated off-line using WRF output, we could deliver to the server just the ETo and precipitation values for accumulation purposes. However, the GDC application requires daily maximum and minimun temperatures and so we deliver these as well, in netCDF format. All three domains for each sector are available on a daily basis from here. Use the netCDF programs "ncdump" and "ncview" to inspect the headers and contents.
Global Forecast System
Agrineer's implementation of the WRF program uses the Global Forecast System (GFS) forcing files as input. These are files generated by NOAA's National Centers for Environmental Information (NCEI),which are used to force data interpolations to defined values at certain times, projected into the future at six hour intervals. Once a projected time is reached, the forcing file is regenerated using real data instead of projected data and a new set of projected files are generated, and the process repeats itself. This project uses the re-analyzed files at six-hour intervals for historical evaluation. The resolution of the data is 2.5 x 2.5 degrees, and the gridded data is interpolated down to a 0.25 X 0.25 degree resolution to provide finer detail. The data is retrieved from here where the folders are date and interval dependent. To retrieve re-analyzed data for 2017/03/03 interval 00, for example, go to gfs.2017030300, then find gfs.t00z.pgrb2.0p25.f000.
Platform
Agrineer's WRF platform is Linux Mint as the operating system on an 8-core 64-bit AMD based computer, compiled under GNU gfortran with MPICH parallel implementation. The Python language is used to integrate all of the working parts, from retrieving input files to uploading results to the server. The dynamic solver model used in WRF is the ARW (Advanced Research WRF) core.
The computer platform described can calculate 10 sectors per day, with rest periods for maintenance. For redundancy, Agrineer operates two WRF platforms in separate long distance locations.
WRF Docker Container
A Docker container package is provided which generates WRF data used in the GDC and SME tools. Third party data contribution for specific sectors can be done with this package and is suitable for compute servers and multi-core desktops alike. The container package can be downloaded here. Registration with Agrineer.org is required for data contributions used in the GDC and SME tools.
