CC BY
16
RUSSIAN JOURNAL OF EARTH SCIENCES, VOL. 10, ES3007, doi:10.2205/2007ES000227, 2008
Information-measuring complex and database of mid-latitude Borok Geophysical Observatory
S. V. Anisimov,1 A. Chulliat,2 and E. M. Dmitriev1
Received 1 April 2008; accepted 10 April 2008; published 17 April 2008.
[1] The measuring complex and Database of Borok Geophysical Observatory (branch of the Schmidt Institute of Physics of the Earth, Russian Academy of Sciences) are considered as the example of the application of information technologies in geophysical observation. The measuring complex of the observatory and the basic types of observation are described.
Specificity of information technologies for all stages of working with geophysical data, including logging, processing, storage and access to them, are discussed. Principles of design of digital data logging systems and networks, their architecture and the software are presented. Methods and programs of raw data processing (both to store and to simplify access to the data), architecture and software of the database are explained.
Various methods to access geophysical data, caused by various methods of data storage are characterized. Access via Internet, as the most universal, operative and convenient for users, is considered. The experiences of Borok Geophysical Observatory in the international networks of magnetic observatories are described. INDEX TERMS: 1555 Geomagnetism and Paleomagnetism: Time variations: diurnal to decadal; 1560 Geomagnetism and Paleomagnetism: Time variations: secular and longer; 1594 Geomagnetism and Paleomagnetism: Instruments and techniques; 3304 Atmospheric Processes: Atmospheric electricity; KEYWORDS: Borok Geophysical Observatory, INTERMAGNET, magnetic observatories, information technologies in geophysical observation, geomagnetic data.
Citation: Anisimov, S. V., A. Chulliat, and E. M. Dmitriev (2008), Information-measuring complex and database of mid-latitude
Borok Geophysical Observatory, Russ. J. Earth. Sci., 10, ES3007, doi:10.2205/2007ES000227.
Introduction merical modeling problems demanded rather small data vol-
umes, and digitizing could be carried out manually directly ahead of modeling. Usually the data were stored as the pa-
[2] Continuous geophysical observations form experimen- per plots or tables. Afterwards data volumes increase and tal base of geophysical researches in the evolution, the struc- mathematical models become more complicated, leading to ture and the modern state of the Earth, its atmosphere, hy- requirement of digital data logging. Devices for reading ana-
drosphere and all geophysical shells. During last decades in log information (digitizers, scanners) and for analog signal such researches the information and network technologies are digitizing (analog to digital converters) have been created. very imp°rtant. Computer processing of geophysical data [4] Thus, data digitizing began to be considered as a
includes their representation in a standard form (format- part of data processing. Now main geophysical observato-
ting), statistical processing (making of statistical character- ries record the geophysical information to digital media. So istics), visualization of the geophysical information (plots, researchers deal with the raw geophysical data already prediagrams maps) numerical modeling of the geophysical en- sented in a digital form. There is a necessity to maintain vironment state and geophysical processes. a11 these tasks long-term storage of the digital geophysical data, to design demand representation of the initial data in a digital form. universal data formats and to adjust formats of the different
[3] The requirement for digitizing of the data from vari- data
ous analog medium appears from problems of mathematical [5] Today the data processing context extends, includ-
and numerical modeling of geophysical processes. First nu- ing rejection, interpolation, filtration, decimation of the raw
—-------------------- data as well as their elementary statistical analysis, provided
1Borok Geophysical Observatory of Shmidt Institute of u u • i u + mu j- •+ i • +
by many geophysical observatories. Thus digital registra-Physics of the Earth, Borok, Yaroslavl’ Region, Russia ¿jots &
2Institut de Physique du Globe de Paris - CNRS, Paris, France tion allows automating the raw data processing, to reduce
its time and to obtain processing results in near real time
Copyright 2008 by the Russian Journal of Earth Sciences. mode.
ISSN: 1681-1208 (online) [6] The further development of digital technologies, ap-
Figure 1. SAMNET magnetic stations.
pearance of personal computers and expansion of their application area stimulated also their usage in geophysical researches. It concerns, first of all, to visualize the data, to construct various plots, diagrams and maps. Effective visualization of data processing results is considered as the necessary part of geophysical data processing.
[7] The application of information technologies to the geophysical observations has resulted in occurrence of the geophysical databases, including raw geophysical data as well as data processing results and providing to users various opportunities for the data access, analysis and visualization. Creation and maintenance of these databases have undertaken both geophysical services, and large geophysical observatories. Their activity, in particular, has resulted in creation of the World Data Centers (WDCs), collected and provided the data on geophysical, solar-terrestrial and environmental problems (http://www.ngdc.noaa.gov/wdc/wdcmain.html). One of them - The World Data Center on Solar-Terrestrial Physics is located in Moscow (http://www.wdcb.ru/stp/ index.ru.html).
[8] Firstly geophysical databases were distributed on digital hard media. Working with the geophysical observation results has qualitatively changed with occurrence of computer networks. Internet allows not only to provide researchers the convenient access to geophysical databases, but also to create the allocated databases and the international
data logging networks to collect the geophysical data in near real time mode. Now the regional and global geoinformation systems are created to provide operative complex access to the various geophysical data, collected from various sources. The expert systems are developed to analyze the complex geophysical information, including ones, working in near real time mode. These systems are designed with usage not only data, but also experiences of geophysical observatories in the application of information technologies and telecommunications to the geophysical observations.
[9] Not only geophysical observatories but also the international geophysical networks provide access to the geophysical data via Internet. Borok Geophysical Observatory works in the Sub-Auroral Magnetometer Network (SAMNET) since 1998 and in the International Real-time Magnetic Observatory Network (INTERMAGNET) since 2004 [Chulliat and Anisimov, 2007].
[10] The SAMNET is one of the national scientific programs of the Great Britain, created in 1987 with the purpose of the decision of problems of solar-terrestrial physics. SAMNET functioning is provided with Department of Communication Systems at Lancaster University. 14 magnetic stations of the Great Britain, located in Faeroess, Sweden, Norway, Finland, Iceland and Russia, participate in the program (Figure 1). Geomagnetic field variations are registered with sampling rate 1 s at each station. The absolute time
Figure 2. INTERMAGNET magnetic observatories.
is provided by the global position system (GPS). The data collected within the framework of SAMNET program, are free using in the scientific purposes.
[11] The INTERMAGNET is the global network of observatories, monitoring the Earth’s magnetic field. The INTERMAGNET program exists to establish a global network of cooperating digital magnetic observatories, adopting modern standard specifications for measuring and recording equipment, in order to facilitate data exchanges and the production of geomagnetic products in close to real time. In 2007, there are 107 magnetic observatories in 38 countries, operated in the framework of INTERMAGNET (Figure 2). The observatories and participating institutes operatively transfer the geomagnetic data to the regional geomagnetic information nodes by satellite, computer and telephone networks, using standard formats of INTERMAGNET. Regional geomagnetic information nodes are the collection and dissemination points for real-time data within INTERMAGNET. The participating observatories and institutes have free access to the all collected geomagnetic data.
[12] This paper describes the results and perspectives of the application of information technologies in geophysical observations at the middle-latitude Borok Geophysical Observatory [Anisimov and Dmitriev, 2003a, 2003b, 2003c; Anisimov et al., 1999, 2002], (S. V. Anisimov and E. M. Dmitriev, 2005, http://www.scgis.ru/russian/cp1251/ h_dgggms/1-2005/inform-1.pdf; S. V. Anisimov et al., 2000, http://www.scgis.ru/russian/cp1251/h_dgggms/3-2000/ anisimov.htm; S. V. Anisimov et al., 2001, http://www.scgis. ru/russian/cp1251/h_dgggms/4-2001/anisimov.pdf). Borok Geophysical Observatory realizes all kinds of geomagnetic measurements (geomagnetic pulsations, variations of a geomagnetic field, and absolute measurements of a magnetic field of the Earth). The observations of various geophysi-
cal fields include the measurements of atmosphere electricity (atmosphere electric field and current), telluric currents, atmospheric pressure variations, meteorological parameters and also Doppler ionosphere sounding. The scheme of the main measuring complex of the observatory is presented on Figure 3. The data logging, processing, storing and access are applied by the information technologies.
Geophysical Observations in Borok Geophysical Observatory
Geomagnetic Field Observations
[13] One of the most advanced directions of geophysical researches is observation of the Earth magnetic field. The geophysical observatory can be considered like magnetic if there are carried out long-term continuous observations of geomagnetic field variations, regular absolute measurements, processing and analysis of raw data. Continuous digital registration of geomagnetic field in the wide frequency range with data presentation in absolute physical scales and exact absolute time is carried out on modern magnetic observatories. Daily files of geomagnetic field variations, presented as digital amplitude-time series adjusted by the results of absolute measurements, are formed. The modern level of measuring equipment, digital technologies and satellite systems of global positioning allows making accuracy of the main geomagnetic field measurement about 1 nT.
[14] The middle-latitude Borok Geophysical Observatory observations include digital registration of three components of geomagnetic field, absolute measurements, geomagnetic
Figure 3. Main measuring complex of Borok Geophysical Observatory.
field variations, ultra low frequency geomagnetic pulsations. The geographic location of the observatory, the geomagnetic field registration received continuously within several decades, a low level of electromagnetic noise result in geomagnetic data usage by Russian and foreign geophysicists.
[15] Geomagnetic field on Borok Geophysical Observatory are registered within the framework of international program INTERMAGNET (http//www.intermagnet.org). The observatory acts in the global network of digital magnetic ob-
servatories INTERMAGNET according to rules and standards of global network of geomagnetic observations [St-Louis, 2007]. The measuring equipment includes scalar and vector magnetometers and devices for absolute measurements, including the precision proton magnetometer and Declination/Inclination fluxgate-magnetometer.
[16] Scalar proton magnetometer SM-90R (produced by GEOMAG, France) measures total magnetic field and has the following basic characteristics:
• sensitivity 0.05 nT;
• dynamic range 32000-70000 nT;
• frequency range 0-0.3 Hz;
• resolution 0.1 nT;
• accuracy 1 nT.
[17] Vector three-component fluxgate-magnetometer VM-391 (produced by IPGP, France) measures three components of magnetic field and has the following basic characteristics:
• sensitivity 0.1 nT;
• dynamic range ±2500 nT;
• frequency range of 0-0.1 Hz;
• resolution 0.1 nT;
• temperature stability 0.25 nT/°C;
• long-term stability 5 nT year-1.
[18] Absolute measurements of a geomagnetic field are carried out by portable single-axis fluxgate vector magnetometer, type Mag-01H, installed on the steel-free Zeiss theodolite. Magnetometer Mag-01H (produced by Bartington Instruments), provides precision measurements of the direction and intensity of static and slowly varying magnetic fields from 0.1 nT to 2 mT with 0.1 nT resolution.
[19] The geomagnetic field variations are registered by the fluxgate-magnetometer of SAMNET network (http://www. dcs.lancs.ac.uk/iono/samnet/). The fluxgate-magnetometer measures three components of magnetic field with 1 s sampling rate and 0.1 nT resolution.
[20] Ultralow-frequency geomagnetic field pulsations are registered by the induction magnetometers, produced in the Geoelectromagnetic Monitoring Laboratory of Borok Geophysical Observatory (http://borok.adm.yar.ru/gemm/ index.html).
[21] The magnetometers include induction sensors with permalloy cores, measuring amplifiers and blocks of filters. The linear frequency characteristic of magnetic sensors allows expanding a dynamic range of measurements owing to indemnification of spectral heterogeneity and a wide range of changes of amplitudes of geomagnetic pulsations. Transfer factor of the active induction sensor is about 1 V/(nT Hz) with the built-in amplifier. The basic characteristics of induction magnetometers are
• a frequency range 0.001-3.0 Hz;
• noise level 0.5 pT/Hz1/2 on frequency f = 1 Hz.
Telluric and Air Electricity Measurements
[22] Measurements of telluric currents are necessary to study the structure of ULF geomagnetic pulsations, to produce remote diagnostics of magnetosphere plasmas, to research magnetosphere resonant features, to design the methods for the electromagnetic forecast of the abnormal geophysical phenomena, to develop methods of magneto-telluric
sounding. High-sensitive, precision and noise stable equipment is used to measure the telluric currents.
[23] Variable part of the potential difference between the fixed points of ground surface is measured by pairs of lead electrodes, located along the magnetic meridian, along the magnetic parallel and vertically. Observable components of telluric electric field are determined from the potential difference between the corresponding pair of the electrodes referred to the fixed distance between them.
[24] The main difficulty of registration of telluric currents consists in maintenance of the high noise stability of the equipment. Authentic results are guaranteed by installation of electrodes in region with a very small industrial electromagnetic noise level, far from power plants, the mass transit electrified lines and large towns. Thus the magneto-telluric measuring complex should be installed in areas with a homogeneous geological and geoelectric structure, and also with a quiet relief. The territory near location of Borok Geophysical Observatory completely meets the specified requirements.
[25] The lead sheets, used as electrodes for measurement of telluric current horizontal components, are buried on depth 1.0 m with distance between electrodes 300 m. Electrodes for measurement of telluric current vertical component are located in the vertical hole with 400 m depth.
[26] To suppress the in phase noise in the measuring channel of telluric currents the symmetric circuit with a differential input is used. Technical parameters of the measuring equipment are
• threshold sensitivity ~0.01 ^V m-1;
• frequency range 0.001-3.0 Hz.
[27] The atmospheric electric field is measured by electrostatic fluxmeter (field-mill), convert constant or slowly varying electric field to the variable one by means of periodic screening of measuring electrodes by rotated and grounded screen. The field-mill, installed in the observatory measuring complex, has been designed specially for long continuous observations. The field-mill design provides the constant exposed area of measuring electrode and allows increasing sensitivity by the differential measurement method. The basic characteristics of field-mill are
• dynamic range for constant electric field 80 dB;
• noise level 0.1 V/Hz1/2 on frequency f = 1 Hz;
• frequency range of 0-5 Hz;
• sensitivity 1 V m-1.
[28] The density of an atmosphere vertical electric current of is measured by means of “current collector”. The basic part of installation is the ring wire antenna with 300 m diameter. The effective height of the antenna is 4.5 m, the effective area is about 2500 m2.
[29] The regular error of the antenna effective area calculation does not exceed other random errors of air electric measurements and makes ~10%. For working antennas the calculation of the effective area can be adjusted by measurement of real antenna capacity. Thus the dynamic and static
effective areas are considered to be approximately equal, that is true for the advanced turbulent electrode effect, influenced up to the effective height of the antenna.
[30] The vertical electric current density of an atmosphere is measured by the methods of voltage decreasing on the fixed stable resistance. The block of filters in the channel of an atmospheric current density measurement is similar to those, used in the measuring complex for registration of other stationary geophysical fields. The basic characteristics of the measuring equipment are
• sensitivity -10-13 A m-1;
• dynamic range ~10-13-10-9 A m-1.
Doppler Radio Sounding of the Ionosphere
[31] High-frequency Doppler method for ionosphere researches is based on observation of frequency variations of the radio wave reflected from ionosphere inhomogeneities, changing in time and in space. Thus changes of an ionosphere state can be caused by solar geophysical factors as well as by atmosphere and seismic events from natural and artificial origins. Propagation of the radio wave is accompanied by frequency deviation caused by changes in physical properties of media and in the geometric trajectory or an altitude of reflection.
[32] The high-frequency Doppler method consists in comparison between the frequency of the continuous radio signal, reflected by the ionosphere, and the frequency of the stable basic generator. Usage of the basic source enables to apply the device both for vertical and inclined sounding. The basic generator frequency is shifted on some hertz from the transmitter frequency to detect Doppler shift.
[33] The equipment resolution, or its sensitivity, depends on the frequency stability of the heterodyne receiver and carrier wave of the radio transmitter. The highly stable broadcasting radio station working in a short wave range is used as radio transmitter. The radio receiver basic generator is the quartz generator with proportional thermostat system of the resonator, providing the frequency stability about 10-8. So the resolution of the measuring complex on frequencies about 10 MHz is provided at 0.1 Hz level. With decreasing of a radio frequency this value decreases proportionally. Main parameters of the measuring are
• carrier wave frequency 4.625 MHz; receiver resolution 0.1 Hz.
Meteorological Measurements
[34] Meteorological measurements on modern observatories are carried out by the digital meteorological stations. The professional digital meteorological station WS-2500, providing continuous measurements of air temperature and humidity, wind direction and velocity, atmospheric pressure and precipitation, is included into the measuring complex of
Borok Geophysical Observatory. Meteorological station WS-2500 consists of the base station, equipped with the power adapter and the cable (5 m) to communicate with the personal computer, set of temperature and humidity sensors (up to 9), the wind sensor and the precipitation sensor. The station is equipped with the display and to transfer the measured parameters to the personal computer via COM-port in RS-232 format. Power supply of the meteorological station is carried out via the power adapter. Power supply of the sensors is carried out from solar batteries. The obtained data are transferred from sensors to base station by the radio channel on frequency of 433 MHz.
[35] The digital ultrasonic meteorological stations, using contactless methods to measure wind velocity and air temperature, are applied in the atmosphere physics researches, including atmosphere dynamics. In measuring complex of Borok Geophysical Observatory this kind of meteorological stations is presented by the automatic ultrasonic complex “Meteo-2”, produced by Institute of Atmosphere Optics of the Russian Academy of Science. The complex measures temperature, velocity and direction of the wind (air streams), relative humidity, atmosphere pressure. The complex allows to estimate also parameters of air turbulence. The raw data are transferred to the personal computer via the cable as a digital code in RS-232 format.
[36] The complex “Meteo-2” includes the ultrasonic measuring block, the block of humidity and pressure sensor, the power unit and personal computer (operation system Windows 95/98/2000/NT) with the special software. Both measuring blocks are placed directly in the air media. They can be installed on a mast for measurements in a free atmosphere. The power unit is installed near a computer.
[37] The ultrasonic measuring block includes ultrasonic system and the electronic module to convert measured values to a digital format and to transfer them to the computer. The ultrasonic system is designed as two tubular metal rings, installed vertically and orthogonal to each other, with 8 ultrasonic sensors fixed on the rings. The measurements of air temperature and three orthogonal component of wind velocity are based on functional dependence of sound group velocity from the specified meteorological parameters. The software calculates sound wave group velocities for four directions of its propagation and then meteorological values are calculated.
[38] The humidity sensor sensitive element is serial sorption-capacitor element with the sensitive layer dielectric permeability depending on air humidity. The atmosphere pressure sensor is electric strain gage, with the output voltage depending on atmospheric pressure. So the ultrasonic complex “Meteo-2” allows to measure:
• air temperature from -40 up to +50° C, with accuracy ±0.3°;
• wind velocity from 0.2 up to 30 m s-1, with accuracy
±0.2 m s-1;
• air relative humidity from 10 up to 98%, with accuracy ±5
• atmosphere pressure from 80.0 up to 106.7 kPa, with accuracy ±0.3 kPa.
[39] For continuous measurements of infrasonic variations of atmosphere pressure the high-sensitivity liquid microbarograph, designed and produced by Institute of atmosphere physics of the Russian Academy of Science, is used. The microbarograph basic characteristics are
• sensitivity 16 mV/(dyne/cm2);
• electronic circuit drift less than 5 mV/°C per half hour.
Information Technologies in Borok Geophysical Observatory
[40] The data logging system is the basic unit to any geophysical observations. It is an allocated computer to carrying out logging and storing of the data from the sensors of geophysical fields.
[41] The modern data-acquisition equipments, as a rule, have hardware and software interface of measuring devices with a personal computer. The measurement data can be transformed to the digital format directly in the measuring device as well as in the special box of analog-to-digital converter (ADC). Then measurement data come to a personal computer already in a digital format through RS-232 or USB ports. The measuring equipment software provides the data visualization and storage to the hard disk. The wide spectrum of the specialized data logging systems, including systems designed to work in field, have applications in the industry as well as in scientific researches.
[42] So, the automatic INTERMAGNET magnetic observatory uses the specialized personal computer ENO II (manufactured by IPGP) with the 1386/40 MHz processor as the data logging system. ENO II device is intended for digital registration of geomagnetic data, coming from scalar and vector magnetometers. The sampling rate of the output data is 1 minute and will be increased to 1 second within the next year. The synchronization with the universal time (UT) is carried out by the built-in global position system (GPS).
[43] The meteorological station WS-2500 is connected to a personal computer via RS-232 socket. The meteorological station software works under the Windows operating system (OS), allowing the operator to get the data from meteorological sensors and to display the current information. The current meteorological data are stored on the hard disk in an internal digital format.
[44] The complex “Meteo-2” provides the measurement data to the personal computer via COM-port. Then data processing is curried out by the corresponding software. The data logging software works under the OS Windows in an asynchronous mode, using interruptions of the COM-port adapter. The background mode running provides the measurement data obtaining, processing and storing on the hard disk. The measurement processes are visualized on the display.
[45] However, the specialized data logging system software very often is badly adapted to tasks of the continuous observation data digitizing. These tasks assume working in
continuous mode, the data storing on the replaceable media or the data receiving via the local access network, continuous synchronization of the registration with universal time. Besides the data logging systems have to provide digital data from the already working analog recording equipment. The inexpensive built-in ADC allows easy transforming the common personal computer to the data logging system.
[46] The choice of ADC characteristics is caused, first of all, by characteristics of the analog data-acquisition equipment. In particular, the number of measured parameters determines the minimum number of ADC input channels, a dynamic range and accuracy of measurement sensors - the ADC word length, frequency characteristics - the sampling frequency on analog inputs. If the signals with various characteristics have to be recorded on the same computer, then the ADC characteristics should get out from the most sensitive sensor and widest frequency band. The requirements to the personal computer on which the ADC is installed, depend from the ADC type. Modern ADC has the built-in timer, the big input buffer to data accumulation and fast co-processor. So this ADC can be installed even on the ordinary personal computers to use them as enough fast and effectively working data logging systems.
[47] A number of specific problems appear in designing the data logging system software for continuous geophysical observations. The main task is exact synchronization of registered data with universal time. Usually precision GPS serve this purpose in the modern data logging systems. In particular, the UT synchronization via GPS is used by the equipments, operated in networks INTERMAGNET and SAMNET.
[48] The main data logging system of measuring complex of Borok Geophysical Observatory, logging the data from all recording analog equipment, is synchronized with universal time by signals from precision quartz clocks AKV-2M. The quartz clocks provide a relative error ±3-10-9 and an opportunity to correct the current time by a radio signal with error 3-10-3 s. Because the sampling rate of the main data logging system is 10 Hz, this accuracy is quite enough. The program of registration uses time code signals generated by quartz clocks, coming to digital inputs of ADC. The durations of pulses, following with frequency 1 Hz, (the pulse front is corresponded to the beginning of the second), contains the information about the current date, hour and minute. The logging software decodes the time code and exposes sets the calculated current time to the computer clocks, providing the synchronization of the registered data with the universal time.
[49] The method of interrogation of ADC input channels depends on the kind of ADC synchronization. The command on data reading can be generated by the logging program, then the logging program should trace the time, coming from computer clocks, continuously. Sometimes it can result in data loss because hardware delays. If ADC has the built-in timer, the data buffer and the opportunity to programming the mode of interrogation of input channels, then the command to interrogation of ADC input channels can be generated by the ADC timer pulses. Because the data collect in ADC buffer, the possible data loss is prevented even for enough long delays of data logging software.
Main (data logging)
begin
InitiateStorage InitiatePlots InitiateADC InitiateClocks while not EOReg do begin
GetDataFromADC
PlotCurrentData
StoreData
ReadKeyboard
end
end
Store (data storage)
procedures
InitiateStorage
StoreData
Display
(visualization)
procedures
InitiatePlots
PlotCurrentData
ReadKeyboard
ADC (ADC & clocks)
procedures
InitiateClocks
InitiateADC
GetDataFromADC
Core (current data)
var
CurrData
StoreData
CurrTime
CurrDigits
TData
TStore
TTime
Byte
Figure 4. The simplified scheme of the program of data gathering.
So, the main data logging system of measuring complex of Borok Geophysical Observatory is equipped with 16-digit and 16-channel ADC, having the built-in data buffer, provides the data preservation in the buffer during 1.6 s for sampling frequency 10 Hz. If there is a precision generator of time marks, a quartz clocks, for example, these marks can be used as pulses triggering the interrogation of ADC input channels. Such algorithm to interrogation of ADC channels is used in the main data logging system of measuring complex of Borok Geophysical Observatory.
[50] Other specific task is the supporting of durable continuous working of data logging systems with an opportunity of data access. Thus there is necessary to transfer the raw data files from the data logging computer hard disk to another computer, making the data processing, storage and archiving. The raw data from the main data logging system are transferred to the data processing computer via the local network. If there is no opportunity to connect data logging system to the processing computer via a local access network, then the raw data is rewritten periodically on the replaceable media with enough volume. So, the data logging software of INTERMAGNET magnetic observatory writes the raw data on a diskette each hour. The data transfer from the magnetic observatory to the processing computer is made manually by replacement of a diskette in the disk drive (the diskette will be replaced by a compact flash memory stick within the next year, thus making this manipulation unnecessary). Thus data logging does not stop.
[51] The simplified scheme of main units of the typical data logging program is presented in Figure 4. In the Core unit the current ADC data and the virtual timer are de-
scribed as global variables. In the Store unit the procedures of initialization and data recording to the media or to the buffer directory, accessible via a local access network, is determined. The ADC unit contains procedures of clock start, and ADC data reading initialization. In the Display unit procedures of initialization and display of the current data plots, as well as procedure of input and processing of operator commands, are realized. In the Main unit the data logging algorithm is realized. The data logging program structure provides easy design and compilation of the program for the any data logging system. It is enough to modify the ADC unit to determine the ADC and synchronization kind, and Store unit to determine the data recording format. Thus the program logic and the operator interface, programmed in Display unit, are kept. The data logging program is written in Pascal language for free distributed interplatform FreePascal compiler that allows using the same source code to compile the program under DOS, Windows and Linux OS.
[52] Each minute the data logging program writes all input data with the information about current time to the buffer directory on the hard disk of the data logging computer. The storage of the minute data files on the hard disk prevents the data loss caused by possible failures of a local access network or a diskette driver. The transfer of data files from the hard disk buffer directory to the other media or to the data processing computer via the data logging network is carried out each hour or each day. The buffer directory of data logging computer is accessible for reading only to guarantee the raw data safety.
[53] All data logging systems, the data processing computers and the database server, served the geophysical database,
Figure 5. A local data logging network.
are connected via the local access network. Thus the local data logging network, providing continuous registration and processing of geophysical data, is created (Figure 5). All computers, included in the data logging network of Borok Geophysical Observatory, work under the Linux OS (http://www.linux.org/), communication between them is carried out by Linux software via the switchboard, functioning independently of the local access network and server of Borok Geophysical Observatory.
[54] The Linux OS installed on the data logging network computers has the following features:
• an open source code, allowing to modify programs for the specific targets;
• a multitasking and the built-in technology of pipeline data processing especially useful for processing of the large files;
• a lot of programs to processing and visualization of the information;
• a free distribution of Linux OS and other UNIX-like OS in the Internet, facilitated the supporting and consultations on the operational system.
[55] Continuous entering of the results of regular observations of middle-latitude geophysical fields to the geophysical database is provided by the main data logging system collected the data from all sensors of geophysical fields. The data logging system carries out the data transformation into the digital format, their initial accumulation and transfer to the data processing computer, logged also the data of SAM-NET and INTERMAGNET magnetometers.
[56] Processing the raw data coming from the data logging computers, including their formatting, averaging and transformation to the physical scale, is carried out on the data processing computer. Two main objectives of the processing are to transform the raw data to the digital format, convenient for data storage and access, and to obtain the data images, convenient for data viewing and analysis by users. So, there are two groups of programs. Formatting programs translate the data received from different sources, into the uniform storage format and store them in the database directories. Visualizing programs produce data plots and spectral-temporal diagrams, storing them as graphic files to allocate on the database web site.
[57] Note, that Linux has the built-in script language Shell and the convenient task manager Cron, allowing to execute during the set moments the data processing scripts, describ-
02781185
Database directory Years
&1997 &2003
Months
---O 01
&12
Days
— ^701
MAIN CHANNELS
00 - geomagnetic pulsations (N-S)
01 - geomagnetic pulsations (E-W)
02 - geomagnetic pulsations (vertical)
03 - telluric current (N-S)
04 - telluric current (E-W)
05 - air electric current (vertical)
06 - air electric field (vertical)
07 - Doppler ionosphere sounding
08 - geomagnetic pulsations (parallel)
09 - telluric current (vertical)
10 - atmosphere pressure
YYYY_MM_DD_HH.CC
YYYY
-Year (1997-2003)
MM
- Month (01-12)
DD
- Day of month (01-31) HH
- Hour (00-23)
CC
- Channel (00-15)
&31
Hour dtta files
-1 2003 1 2 31 00.00 -1 2003 1 2 31 00.15 -1 2003 12 31 23.00
— [il 2003 12 31 23.15
Figure 6. Structure of the database of Borok Geophysical Observatory.
ing all necessary steps to data processing, including running of corresponding programs and commands. Besides it is easy to create “pipeline” of the programs, allows the data from the program output without storing in file go directly to the other program input. Thus, small programs, each aimed to the certain task, are executed sequentially, consistently transforming the data. Final files are created on the hard disk only in the end of the “pipeline” of processing programs. Besides economy of resources, such technology allows to create the effective scripts to process the concrete raw data, based on the small programs, solving the concrete problem of data transformation.
[58] The data processing computer, installed in the data logging network of Borok Geophysical Observatory, executes special programs for the processing of data, coming from the main data logging system, magnetic station SAMNET, automatic magnetic observatory INTERMAGNET and the digital meteorological station. All programs have the features. So, the program for the processing of main logging system
data controls the time code, presented in data, finds skipping in the data series and fills them with the data missing indicator.
[59] After the data pre-processing the programs to transform the data to the physical scales are executed. They transform the hour data files, making simultaneously data averaging and decimation, and store the text data files in database server directories, accessible via Internet.
[60] The programs, providing the data visualization, read the data base files, process them and create corresponding plots and diagrams. As the database files do not contain time marks, the time is defined by the file name, describing the moment of registration beginning. The sample rate is calculated by the processing program trough the number of records in the hour data files.
[61] The visualization of the data of registration of slowly varying parameters, such as geomagnetic field or air electric field, proceeds by the plots of their average values. This problem is solved by the set of programs transmitting the
data to each other by the “pipeline”. The running average of the plotted data is calculated before the visualization. The observed data sets are processed by the plotting program, producing the amplitude-temporal plots.
[62] The visualization of the data of registration of rapidly varying parameters, such as ultra low frequency geomagnetic pulsations, proceeds by the plots of the spectral-temporal diagrams. Thus the processing program gets the data sets and makes their transformation by the fast Fourier transform algorithm instead of the averaging. The visualizing program gets transformed data and makes the spectral-temporal diagrams. All graphic data received during data processing is stored as gif files. The file names have format YYYY_MM_DD_HH_CC.gif, where YYYY - year, MM
- month, DD - day, HH - hour, CC - the channel number.
[63] The database server provides access to the data via the Internet by the local access network of Borok Geophysical Observatory. The database web site, located on the server, contains database manuals, graphic files, data files, interface forms for data requests by database users.
[64] The structure of the data storage directories of the data base server is presented on Figure 6. The data files contain the data from one channel, received during one hour. Names of hour data files have the format YYYY_MM_DD_ HH.CC, where YYYY is year (since 1997), MM is month number (01-12), DD is day of month (01-31), HH is hour (00-23), CC is channel number, which is the unique identifier for each registered parameter. The data file contains the sequence of two-byte records with registered data.
[65] The descriptions of parameters of recorded fields and recording devices are stored in the data base configuration files, including phase-frequency and amplitude-frequency characteristics of input channels, dynamic ranges of the registered fields in ADC bits, coefficients to transform the data to physical scales. The binary format of the data storage (without transformation to physical scales) has a number of advantages. This is unification of the data storage format, the economy of hard disk volume, prevention of loss of the information during data transformations. Thus the archiving and processing programs, making the final data files for
Database web site
----[/^7 Cgi-bin
— &img
---- IZZ7 magnet
---- 2^7 geopuls
— 17^7 electric
— 2^7 weather
____ [1§ index.html
____ gghelp.html
____ 1 copyright ht ml
Web site software Icons, logotypes, graphics Geomagnetic field data ULFgeomagnetic pulsations Air electric field data Meteorological data
Common information User manual
Copyrights, rules of the road
Figure 7. Structure of the web site of Borok Geophysical Observatory database.
the database users, are special for each geophysical field.
[66] The program, processing the data from the magnetic station SAMNET, is executed each day, transforming the data from SAMNET format to the database format. For the data received from SAMNET station, the channels with 16, 17 and 18 numbers are reserved in database. The sample rate in the hour files, corresponding to these channels, is 1 s, like in SAMNET data logging system. The programs, processing meteorological data operate similarly.
[67] Each month the collected geophysical data, stored in the database, are copied on CD-ROM, acting like the basic media to archive database. Copying is made on the archiving computer, having the CD-ROM recorder.
Internet Technologies in Borok Geophysical Observatory
[68] There is most operatively and conveniently for users to access the data via Internet. If users deal with any database parts, distributed on any media, then they can use access and processing programs, designed for the Internet too. Thus, the interface is unified, allowing the user to get data from the database web server via Internet as well as from the database media on the personal computer without Internet access. The free-of-charge software with open sources is used to provide the Internet access and to design the database processing programs. So, the data processing programs of Borok Geophysical Observatory database are written in Pascal and C programming languages supported by Linux. Programs of data requests processing are written in Perl 4 language using the CGI interface. All database software was developed and distributed under the GNU General Public License.
[69] The free access data of the Borok Geophysical Observatory database are located in the data directories of the database server. They include the data on geomagnetic field variations, ULF geomagnetic pulsations, atmospheric electric field vertical component and meteorological data.
[70] The visual information on all geophysical fields, accessible via Internet, is located in corresponding subdirectories (Magnet, Geopuls, Electric and Weather) as the gif files, produced by the data processing software. The separate files contain the common information on the presented data (files index.html), the description of the measuring complex and sensors (sensors.html), database manuals (help.html), and also data request forms (request.html). The visual information files (pictograms, trade marks, plots and spectral-temporal diagrams) are stored in the subdirectories img. The scheme of the database web server is presented on Figure 7. All text information has HTML format, the visual information is presented in gif and jpeg formats. The database home page (http://geobrk.adm.yar.ru:1352/index.html) references to web sites of the organizations supporting the database. There are links to web sites of Russian Academy of Sciences (http://www.ras.ru), the Russian Foundation for Basic Research (http://www.rfbr.ru), the Schmidt Institute of Physics of the Earth (http://www.ifz.ru), Borok Geophysical Observatory (http://wwwbrk.adm.yar.ru) and Geoelec-
Figure 8. Home page and page with site map of the database web site.
tromagnetic Monitoring Laboratory (http://wwwbrk.adm. yar.ru/gemm/index_e.html).
[71] From the home page the web-site visitor can go to database manuals, to rules of the road and to the geophysical data pages, presented the results of measurements of geomagnetic variations, ULF geomagnetic pulsations, atmosphere electric field and meteorological parameters. The home page and page with site map are presented on Figure 8.
[72] All web-site pages have in the left side the menu with links to current level sub-items and logo of Borok Geophysical Observatory, linking to web-site home page.
[73] The menu items referencing to the data viewing (View Data) specify not html files, but the program to generate html-file to view the data. On the first call (without parameters) the program generates the html-file which displays the selected geophysical parameter of the last day or month. Thus the graphic files generated by the data processing programs are used. On next calls from earlier generated files the program gets as parameter the date of the data set to visualize. The program creates the html files displaying the selected geophysical parameter for the selected date. Thus generation of html-pages to view the data is invisible for the user, who goes from one date to another, using the references at the generated html-pages.
[74] To provide data safety and to prevent hacker attacks the raw data files are locked. The data, requested by database users, are formed by the database web-site software. The user requests the data by filling the data request
form, special for each data type; however the algorithm to processing of the data request form is identical to all data types. Data display page and an example of the data request form for the geomagnetic field variations are presented on Figure 9. The user has to fulfill the form fields Time Interval (pointing year, month, day and hour), First Name (pointing user’s name), Surname (pointing user’s family name), E-mail (the e-mail address). After processing of the data request the user receives via e-mail the confirmation on request processing.
[75] The data are sent to the user by e-mail, simultaneously the database administrator is notified on the received data request and the request information is stored in the special log-file on the database server. Thus, the control of data requests with the database administrator is provided.
[76] The database collective users get the data via the routine data receiving. The sent data, their format and periodicity of sending are established by the arrangement with customers. Also depending on requirements and opportunities of customers the data transfer can be provided via e-mail or via the ftp-server. So, the data of geomagnetic field variations are daily transferred via e-mail to the Institut de Physique du Globe de Paris (France) and via the ftp-protocol to the Lancaster University (Great Britain).
[77] Because the slow Internet access to the geophysical observatories, located far from large towns, the mirrors of database site are created to improve the Internet access to the database. The web site of Borok Geophysical Observatory database (http://geobrk.adm.yar.ru:1352) has the
Figure 9. Data display page and an example of the data request form for the geomagnetic field variations from the database web site.
mirror (http://geodata.borok.ru), located on the web server of Network Information Center of Space Research Institute (Moscow). The mirror is the full-function copy of the original web site of the database of middle-latitude Borok Geophysical Observatory, providing the graphic data, programs of generation of display pages, the data request forms, the database manuals and help information. The geophysical data are not transferred to the mirror. When they are requested by the visitor of the mirror, the data requests are sent to the database web site to process by the web-site software.
[78] The access to the geophysical data via Internet can be provided by the International geophysical networks as well as by the geophysical observatories. Borok Geophysical Observatory is the member of the Sub-Auroral Magnetometers Network (SAMNET) and the International near Real Time Magnetic Observatories Network (INTERMAGNET).
[79] Originally the data on magnetic field variations, digitizing by the SAMNET data logging system, were collected on the ZIP-disk and were transferred to the main server of SAMNET by the operator each week. In 2001 magnetic station SAMNET has been included in a local data logging network that has allowed to accumulate the registered data in the database server and to make daily automatic data sending via e-mail to the main server of SAMNET. Since 2003 the observation data are sent to the main server of SAMNET, located in the Lancaster Uni-
versity (http://www.dcs.lancs.ac.uk/iono/samnet) via ftp-report each day.
[80] The standard automatic magnetic observatory INTERMAGNET, installed in Borok Geophysical Observatory, includes vector fluxgate-magnetometer to measure three components of the geomagnetic field, scalar proton magnetometer to measure the magnetic field total value, GPS to synchronize the measurement data with universal time, power supply units, the electronic equipment and the data logging computer. Geomagnetic measurements are carried out with the resolution 0.1 nT. The sampling rate of scalar magnetometer is 1 minute. The data from vector magnetometer are read each 5 s, are filtered by the Gauss smoothing filter, centered in the minute interval. The received minute data of the geomagnetic field observation are collected in the diskette and are transferred to the database server manually each day, whence the data are automatically sent via e-mail to the geomagnetic information node in the Institut de Physique du Globe de Paris (http://obsmag.ipgp.jussieu.fr). Within the framework of INTERMAGNET program Borok Geophysical Observatory staff make the geomagnetic field absolute measurements two times per week. The results of these measurements are necessary to check and correct the data coming from the automatic magnetic observatory. The data of absolute measurements are weekly sent to the Institut de Physique du Globe de Paris via e-mail.
Conclusion
[81] The information technologies in Borok Geophysical Observatory measurement complex are considered. Features of information technologies related to the continuous observations of various geophysics fields are analyzed. The basic approaches to design the hardware and the software to all stages of geophysical data operations (data logging, processing, storage and access) are presented. Algorithms and methods used at the creation and maintenance of the database of middle-latitude Borok Geophysical Observatory are described. The observatory successfully works in the international geomagnetic networks INTERMAGNET and SAMNET. The computer technologies, applied to the routine geophysical observations, result to the database web server, provided the Internet access to the geophysical data.
[82] Acknowledgments. The development of information technologies in Borok Geophysical Observatory became possible only due to financial support of the Russian Foundation for Basic Research (grants no. 99-07-90106, 02-07-90107, 05-07-90262, 08-07-00194).
References
Anisimov, S. V., S. Bakastov, E. Dmitriev, and E. Anisimova (1999), Aeroelectrical Measurements in Geoelectromagnetic
Complete Set of Geophysical Observatory “Borok”, in Proc. 11th Int. Conf. on Atmospheric Electricity, p. 630, NASA, Guntersville.
Anisimov, S. V., E. Dmitriev, E. Anisimova, and S. Bakastov (2002), Information technologies in system of geoelectromagnetic monitoring of Borok Geophysical Observatory RAS, in Materials of the All-Russia Conference “Geophysics on a Boundary of Centuries” (in Russian), p. 71, Regional scientific association on the problems of applied geophysics, Moscow.
Anisimov, S. V., and E. M. Dmitriev (2003a), Information-Measuring Complex and Database of Borok Geophysical Observatory RAS (in Russian), 44 pp., UIPE RAS, Moscow.
Anisimov, S. V., and E. M. Dmitriev (2003b), Aeroelectrical constituant in the database of Borok Geophysical Observatory, in Proc. 12th Int. Conf. on Atmospheric Electricity, p. 693, ONERA/ISP Press, Versailles.
Anisimov, S. V., and E. M. Dmitriev (2003c), The telematic applications in measuring complex and Database of Borok Geophysical Observatory, Abstracts of the WISTCIS Outlook Conference “Information Society Priorities: New Prospects for European CIS Countries”, p. 12, GC RAS, Moscow.
Chulliat, A., and S. V. Anisimov (2007), The Borok INTERMAGNET magnetic observatory, Russ. J. Earth Sci., 10, ES3003, doi:10.2205/2007ES000238.
St-Louis, B., Ed. (2007), INTERMAGNET Technical Reference Manual, vol. Version 4.3, 88 pp., JAGA, Edinburgh.
S. V. Anisimov, E. M. Dmitriev, Borok Geophysical Observatory of Shmidt Institute of Physics of the Earth, Borok, Yaroslavl’ Region, Russia ([email protected])
A. Chulliat, Institut de Physique du Globe de Paris — CNRS, 4 place Jussieu, 75005 Paris, France ([email protected])
