Research group "Earthquake Forecast System based on Seismicity of Japan (EFSSJ)" (the group'ssecretariat is located at Earthquake Research Institute (ERI)) in collaboration with Collaboratory for the Study of Earthquake Predictability (CSEP)
K. Z. Nanjo, N. Hirata, and H. Tsuruoka (ERI) are responsible for the wording of this webpage.
Last modified on September 2010
Japanese version webpage
EFSSJ research group in collaboration with CSEP conducts an earthquake forecast testing experiment for Japan. Main purpose is to evaluate the performance of submitted earthquake forecast models and to better understand the physics and statistics of earthquake occurrence. For this purpose, EFSSJ called for statistics- and physics-based models (read here for details). The EFSSJ also invited methodologies that can be used to measure earthquake forecast accuracy, for examining the validation of the suitability of the collected methodologies. In order to provide a forum for mutual understanding among experiment participants and have detailed discussion on the experiment, we held the International Symposium "Toward Constructing Earthquake Forecast Systems for Japan" on 27 May 2009 at ERI. Here we show a summary of the experiment. This is the 1st experiment, and similar trails will be scheduled.
- Deadline date of model submission: 1 October 2009
- Start date of the experiment: 1 November 2009
- Report on earthquake forecast models registered to the 1st CSEP experiment for Japan (password protected)
- Special Issue of Earth, Planets and Space (EPS) "Earthquake Forecast Testing Experiment for Japan".
Our approach is based on the CSEP testing centers that are modeled after the RELM: Refer the RELM special issue [Seismol. Res. Lett., 78(1), 2007] and the CSEP website. A summary of the game's rules specific to Japan is given below.
- 2.1. Participants and release of test results
- 2.1.1. Modelers: Researchers who submit their earthquake forecast models: each model provides with a prospective earthquake forecast. Models cannot be withdrawn from the test without agreement of the Testing Center.
- 2.1.2. Testing Center: The facility with the CSEP infrastructure at ERI. Each earthquake forecast model is submitted to this Center and is evaluated independently from the modeler. All forecast results are stored: therefore additional prospective tests can be integrated at later stages without compromising integrity of prospective testing experiment.
- 2.1.3. Release of test results: The information on test results is closed to public access. But password-protected web pages will be used to allow a participant community to access this information. An overview of the present testing experiment will be considered for peer-reviewed publication.
- 2.2. Testing regions
- 2.2.1. Region that covers Japan (Fig. 1: eps, jpg) for a depth range d <= 100 km with a node spacing of 0.1°.
- 2.2.2. Region that covers the Japan's mainland (Fig. 2: eps, jpg) for d <= 30 km with a node spacing of 0.1°.
- 2.2.3. Region that covers Kanto (Fig. 3: eps, jpg) for d <= 100 km with a node spacing of 0.05°.
- 2.3. Testing classes
- 2.3.1. 1-day forecast: Forecast models must define a number of earthquakes for each magnitude bin in the magnitude range 4.0 <= M <= 9.0 (0.1 magnitude unit steps) at each node for consecutive 1-day time windows, each starting at midnight in Japanese Standard Time. The magnitude step M = 4.0 covers the magnitude range 3.95 <= M < 4.05. The first forecast time window starts at midnight of 1 November 2009, the second time window does at midnight of 2 November 2009, the third does at midnight of 3 November 2009, and so on.
- 2.3.2. 3-month forecast: Same as (2.3.1) but the time-window length is 3 months. The first forecast time window starts at midnight of 1 November 2009, the second time window starts at midnight of 1 February 2010, the third starts at midnight of 1 May 2010, and the fourth does at midnight of 1 August 2010 in Japanese Standard Time.
- 2.3.3. 1-year forecast: Same as (2.3.1) but the time-window length is 1 year and the magnitude range is 5.0 <= M <= 9.0 (0.1 magnitude unit steps). The first forecast time window starts at midnight of 1 November 2009 and the second time window does at midnight of 1 November 2010.
- 2.3.4. 3-year forecast: Same as (2.3.1) but the time-window length is 3 years and the magnitude range is 5.0 <= M <= 9.0 (0.1 magnitude unit steps). The time window starts at midnight of 1 November 2009.
- 2.4. Future earthquakes that will be forecast: The official bulletin for future earthquakes is the revised JMA bulletin. No declustering is applied to the JMA catalog. In other words, models will be evaluated against observed earthquake data in the JMA nondeclustered catalog using the forecast evaluation methods described below (2.5).
- 2.5. Forecast evaluation methods
- 2.5.1. In spite of the testing regions (2.2) and testing classes (2.3), the official suite of tests used in the CSEP is used. Current CSEP standards include N-, L-, and R-Tests (Schorlemmer et al., 2007) as well as Molchan-, ASS-, and ROC-tests.
- 2.5.2. One proposed evaluation method will be used for the experiment. (A summary of the method is under construction)
- 2.6 Start date of the experiment: 1 November 2009
Deadline date of model submission: 1 October 2009 (Program code or numerical table submission)
Before model submission, each modeler must download (a) and (b), depending on your choice among the testing regions (2.2) and testing classes (2.3).
- (a) GridML: List of nodes (list of longitude-latitude pairs), each at which forecast numbers of events for a predefined magnitude range are computed. For example, the node (122.45 24.15) represents the center of the box with the four edges (122.4 24.1), (122.4 24.2), (122.5 24.2), and (122.5 24.1) for the case of a node spacing of 0.1°.
- (b) ForecastML template: Template of a numerical table containing forecast numbers of events. Each model must output a numerical table in the format according to the template. Because of large file size, you may consider these points.
- Modelers may download a zip file containing one template file that have been compressed to reduce file size.
- Right-click on the "Uncompressed" and choose "Save Link As ..." from the context menu for downloading an uncompressed (original) template file.
- It may take a long time to open the template, when using a XML editor. An alternative is to use a simple word processor such as WordPad.
- Region that covers Japan（2.2.1）
- Region that covers the Japan's mainland（2.2.2）
- Region that covers Kanto（2.2.3）
Each modeler will collaborate with Testing Center's staffs in order to work on installation and function-check of a forecast program code.
A modeler who cannot make his or her code as an open source for the testing classes of 1 and 3-year forecasts (2.3.3 and 2.3.4) must submit a numerical table formatted according to (b). In this case, the modeler must understand that his or her model will be considered as a new participant in a planned experiment, because of no warranty to ensure objectivity on the use of the model same as that used in the present experiment. We thus encourage program-code submission.
The start date of the experiment is given in (2.6). But we accept forecast models for the 1-day (2.3.1) and 3-month (2.3.2) testing classes every three months. Please consult the Testing Center about this option.
The deadline date of evaluation method submission (program-code submission) will be noticed. The corresponding researchers will collaborate with Testing Center's staffs in order to install and function check his or her program code on a case-by-case basis on finding a solution that fits the needs of both sides.
Most of forecast models require data sources such as earthquake catalogs for model development and optimization. If the JMA earthquake data are needed for such purpose, please e-mail the Testing Center (ZISINyosoku-submit@eri.u-tokyo.ac.jp) that can provide modelers with the JMA catalog in the original format and the CSEP format. (JMA earthquake catalog in the latter format is now under preparation.)
The secretariat (N. Hirata, H. Tsuruoka, and K. Z. Nanjo) of the research group "Earthquake Forecast System based on Seismicity of Japan (EFSSJ)" is responsible for the Testing Center at Earthquake Research Institute, University of Tokyo, 1-1-1 Yayoi, Bunkyo-ku, 113-0032 Tokyo, Japan.
- Fig. 1. Testing region (in blue) that covers Japan (2.2.1; eps, jpg). Forecast analysis is done at nodes "+".
- Fig. 2. Testing region that covers the Japan's mainland (2.2.2; eps, jpg). See the caption in Fig. 1.
- Fig. 3. Testing region that covers Kanto (2.2.3; eps, jpg). See the caption in Fig. 1.
