Research group "Earthquake Forecast System based on Seismicity of Japan (EFSSJ)" in collaboration with D. Schorlemmer (SCEC), F. Euchner (ETH), and CSEP Working Group
K. Z. Nanjo, N. Hirata, and H. Tsuruoka (Earthquake Research Institute, University of Tokyo) are responsible for the wording of this web page.
Last modified on May 2009
Research group "Earthquake Forecast System based on Seismicity of Japan (EFSSJ) (tentative title)" (the group's secretariat is located at Earthquake Research Institute (ERI), University of Tokyo) invites statistics- and physics-based models to statistically forecast seismicity in and around Japan based on geophysical data (for example, past and current seismicity data and crustal dynamics data from Global Positioning Systems or GPS) in order to conduct a comparative test study in determining which of these collected models are best for their forecast accuracy function. Main purpose is to evaluate the forecast performance of such submitted models and to better understand the physics and statistics of earthquake occurrence. The EFSSJ expects a researcher that is interested in submitting a model applicable only to a limited region. To accept this type of model, such region is also invited as a testing region in the objective testing framework. Further, the EFSSJ invites methodologies that can be used to measure earthquake forecast accuracy, for the purpose of examining the validation of the suitability of these collected methodologies. This is the 1st earthquake forecast testing experiment. The 2nd trail of this type of experiment will be scheduled next year.
Japan is one of the most earthquake-prone countries in the world. A sound scientific development of earthquake prediction research for Japan is strongly urged upon emerging social demands for protecting people's lives and properties. Therefore the implementation of reliable earthquake forecasts plays an important role on responding to these demands. To enhance this reliability a scientifically rigorous study needs to be carried out to test the usability of proposed earthquake prediction (forecast) methods and to clear the conditions under which these methods are useful.
Existing Japanese research projects aiming at the realization of scientific earthquake prediction have so far focused on better understanding of the mechanism of earthquake occurrence and the development of forecast simulation technologies based on physical modeling of earthquakes. However, the building of a framework for rigorous testing of earthquake forecasts and the creation of testable forecast models have never been central problems to solve for the earthquake related research. This is now a key challenge under the national "Observation and Research Program for Prediction of Earthquake and Volcanic Eruption (2009-2013)". Together with other subprograms associated with forecast simulation analysis of crustal dynamics, the research program on "Earthquake Forecast System based on Seismicity of Japan (EFSSJ) (tentative name)" will tackle the task. Our scientific experiment of testing earthquake forecasts plays an important role on a part of the latter subprogram (EFSSJ).
The "Collaboratory for the Study of Earthquake Predictability (CSEP)" is a global project of earthquake predictability research and has started in 2006 [Jordan, 2006]. This is a successor of the California project "Regional Likelihood Models (RELM)" [Special issue in Seismol. Res. Lett., 78(1), 2007] that implemented an earthquake forecast testing study with several forecast classes. The primary purpose of the CSEP is to develop a virtual, distributed laboratory-a collaboratory-that can support a wide range of scientifically objective and transparent prediction experiments in multiple regional or global natural laboratories. Final goal is to look for the intrinsic predictability of the earthquake rupture process through this development. Currently the CSEP Testing Centers are operating for California, Europe, New Zealand and other regions.
ERI jointed the CSEP, and prepared the Japanese Testing Center in summer 2008 with help from Southern California Earthquake Center (SCEC) and Swiss Federal Institute of Technology (or Eidgenoessische Technische Hochschule: ETH). In this preparation process, a computer system same as that used at SCEC and ETH for the CSEP data processing was installed at ERI. Using this computer system, we started conducting the experimental study for Japan [Tsuruoka et al., 2008]. The testing region is given in Fig. 1 (pdf, jpg). Currently a set of three 1-year smoothed-seismicity models based on different earthquake-generation hypotheses is under test: Triple-S, JALM, and RI models (Left, middle, and right panels, respectively, in Fig. 2 (pdf, jpg)). Our testing approach is a prospective way that the performance for registered models is assessed through forecasting truly future earthquakes. This prospective performance assessment makes the tests more rigorous and objective because the forecast results are unknown when the scientific prediction experiments are started in operation. The three models are evaluated against the authoritative observed data maintained by Japan Meteorological Agency (JMA) with the official suite of tests used in the CSEP experiments. Currently 3 evaluation tests are used in the Testing Center at ERI: N (number of events) Test, L (log likelihood) Test, and R (log likelihood ratio) Test [Schorlemmer et al., 2007]. Additional tests expect to be integrated in near future.
Using the Testing Center, we formally start conducting the comparative forecast experimental research in 2009 for Japan. For this research, we invite earthquake forecast models. The experiment objectively evaluates the performance of earthquake forecast models. This objective approach seeks to provide the progress of sophisticating the registered physics- and statistical-based models, which will contribute to the forecast accuracy improvement of these models. Testing classes are 1-day, 1-year, and 5-year forecasts. The Testing Center also expects a researcher that is interested in submitting a model applicable only to a limited region. To accept this type of model, such region is invited as a testing region. To make an international contribution to the field of earthquake predictability research, we also invite forecast evaluation methods that are different from the commonly accepted official suite of tests used in the earthquake predictability research field.
Our approach is based on the CSEP testing centers that are modeled after the RELM: Refer the RELM special issue [Seismol. Res. Lett., 78(1), 2007] and the CSEP website. A summary of the game's rules specific to Japan is given below.
Each model must forecast seismicity rates for locations given in the list (3.1) and output a numerical table containing forecast rates in the format according to the ForecastML (3.2).
If the JMA catalog and other data such as GPS data are needed for model optimization, contact the Testing Center.
Modelers must download the application form (English, Japanese) to give notice of forecast model proposal, fill in the form, and send it back to the Testing Center before the deadline date (6.1). After this notice, each modeler will collaborate with Testing Center's staffs in order to work on installation and function-check of his or her model code. A modeler who cannot be made his or her code as an open source for the testing classes of 1- year and 5-year forecasts (2.3.2 and 2.3.3) must contact before the deadline date (6.1) so that a numerical table formatted according to (3.2) will be accepted.
Download the application form (English, Japanese) to give notice of testing region proposal, fill in the form, and send it back to the Testing Center before the deadline date (6.2). After this notice, each will work with the Testing Center on a case-by-case basis to find a solution that fits the needs of both sides.
Download the application form (English, Japanese) to give notice of evaluation method proposal, fill in the form, and send it back to the Testing Center before the deadline date (6.3). After this notice, each corresponding researcher and the Testing Center will work on a case-by-case basis on finding a solution that fits the needs of both sides.
Here is a set of the application form and deadline date for the three proposals above (3-5) for the 1st earthquake forecast testing experiment. After receiving the form from each applicant, Testing Center's staff will contact the applicant.
The secretariat (N. Hirata, H. Tsuruoka, and K. Z. Nanjo) of the research group "Earthquake Forecast System based on Seismicity of Japan (EFSSJ)" is responsible for the Testing Center at Earthquake Research Institute, University of Tokyo, 1-1-1 Yayoi, Bunkyo-ku, 113-0032 Tokyo, Japan.
We thank D. Schorlemmer (SCEC), F. Euchner (ETH), and CSEP Working Group for installing an earlier version of the CSEP infrastructure operated at ERI.
Fig. 1. The testing region that is currently used for the 1-year forecast class and will be used for 2.2.1. The grid spacing is 0.1 degree. From Tsuruoka et al. [2008]. Download (pdf, jpg)
Fig. 2. The 1-year smoothed-seismicity models running currently in the Testing Center: Triple-S (Left), JALM (Middle), and RI (Right). The color bar in each panel shows expectations for the annual number of earthquakes. Note that the scale varies from one panel to the other. Download (pdf, jpg)
