НОВЫЙ МЕТОД ИССЛЕДОВАНИЯ ПОВЕРХНОСТИ: ВОССТАНОВЛЕНИЕ СВЕРХВЫСОКОГО РАЗРЕШЕНИЯ ОРБИТАЛЬНЫХ СНИМКОВ МАРСА, ВЫПОЛНЕННЫХ ПУТЕМ МНОГОКРАТНОГО ПРОХОДА

Тао Ю; Мюллер Ян-Петер

АСТРОНОМИЯ

Ю Тао, Я.-П. Мюллер

а

НОВЫЙ МЕТОД ИССЛЕДОВАНИЯ ПОВЕРХНОСТИ: ВОССТАНОВЛЕНИЕ СВЕРХВЫСОКОГО РАЗРЕШЕНИЯ ОРБИТАЛЬНЫХ СНИМКОВ МАРСА, ВЫПОЛНЕННЫХ ПУТЕМ МНОГОКРАТНОГО ПРОХОДА / A NOVEL METHOD FOR SURFACE EXPLORATION: SUPER-RESOLUTION RESTORATION OF MARS REPEAT-PASS ORBITAL IMAGERY

.n

Аннотация. Высокая степень разрешения снимков поверхностей высоко ценится международным сообществом планетологов, заинтересованных в улучшении понимания процессов формирования поверхностей планет. Однако, учитывая различные физические ограничения инструментов визуализации, связанных, главным образом, с шириной полосы пропускания каналов передачи данных, необходимо найти компромисс между степенью пространственного разрешения и пропускной способностью. Однако, даже в случае использования оптических каналов связи, вряд ли будущие системы визуализации окажутся способны показывать для большинства планетарных тел, таких как Марс, детали размером меньше, чем 25 см. В настоящей статье предложена новая технология потокового восстановления сверхвысокого разрешения изображений Gotcha-PDE-TV (GPT), которая использует дополнительную, так называемую «субпиксельную» информацию о небольших искажениях растра, связанную с флуктуацией зенитного угла зрения при последовательных съемках. Благодаря этой методике, использование избыточной информации, содержащейся в необработанных снимках, позволяет повысить степень разрешения итоговых изображений. В работе продемонстрирована оптимальность использования предлагаемой технологии для восстановления сверхвысокого разрешения планетарных изображений на примере обработки данных проекта получения изображений с высоким разрешением (25 ст High Resolution Imaging Science Experiment - 25 cm HiRISE). В рамках этого эксперимента были, в частности, получены изображения кратера Гусева путем его многократного пересечения марсоходом MER-A (Mars Exploration Rover - MER-A). В результате обработки по предложенной технологии 8 изображений, снятых с разрешением 25 см, удалось восстановить снимки этой зоны с разрешением 5 см.Оценка достоверности воссозданных изображений со сверхвысоким 5-сантиметровым разрешением была проведена с использованием снимков поверхности, полученных тогда же навигационной камерой (Navcam) марсохода, путем сопоставления ориентиров на обоих наборах снимков.

Ключевые слова: Марс, Орбитальные снимки, Многократный проход, HiRISE, Сверхвысокое разрешение, Планетоход, Космическая наука, Кратер Гусева, Планетология, Поверхность планеты.

Abstract. Higher resolution imaging data of planetary surfaces is considered desirable by the international community of planetary scientists interested in improving understanding of surface formation processes. However, given various physical constraints from the imaging instruments through to limited bandwidth of transmission one needs to trade-off spatial resolution against bandwidth. Even given optical communications, future imaging systems are unlikely to be able to resolve features smaller than 25 cm on most planetary bodies, such as Mars. In this paper, we propose a novel super-resolution restoration technique, called Gotcha-PDE-TV (GPT), taking advantage of the non-redundant sub-pixel information contained in multiple raw orbital images in order to restore higher resolution imagery. We demonstrate optimality of this technique in planetary image super-resolution restoration with example processing of 8 repeat-pass 25 cm HiRISE images covering the MER-A Spirit rover traverse in Gusev crater to resolve a 5 cm resolution of the area. We assess the "true" resolution of the 5 cm super-resolution restored images using contemporaneous rover Navcam imagery on the surface and an inter-comparison of landmarks in the two sets of imagery.

Keywords: Planetology, Gusev crater, Space Science, Rover, Super-resolution, HiRISE, Repeat-pass, Orbital images, Mars, Planetary Surface.

DOI: 10.7256/2453-8817.2017.2.22876 65

на Я

м г

M ffl

1. Introduction

Higher spatial resolution imaging data is almost always considered desirable to the international community of planetary scientists interested in improving understanding of surface formation processes. The higher the resolution, the closer the images are to the types of resolution used by geologists to interpret such processes on Earth. For example, studying an area on Mars using 12 m panchromatic HRSC (High Resolution Stereo Camera - http://sci.esa.int/mars-express) allows you to be able to visualise the geological context whilst 6 m CTX (Context Camera - http://mars.jpl.nasa.gov/mro) images allows you to see important mineralogical and geomorphological information which you cannot easily see in HRSC and finally for a tiny percentage of the Martian surface (~1%), 25 cm HiRISE (High Resolution Imaging Science Experiment - http://mars.jpl.nasa. gov/mro) allows you to see details of surface features such as fine-scale layering. However, the resolution of 25 cm is not high enough to view features such as individual rocks with diameters less than 0.75 m or see the types of sedimentary features that MSL (Mars Science Laboratory - http://mars.nasa.gov/msl) Curiosity has found in rover-based imagery. Nevertheless, with various physical constraints from the imaging instruments themselves, not the least of which is "launch mass" and volume, one needs to be able to trade-off spatial resolution and bandwidth for any remote sensing system.

This suggests that even with optical communications, future imaging systems are unlikely to be able to л resolve features smaller than 25 cm given constraints on telescope mass and size. This is also the experience for civilian Earth Observing satellites where the highest spatial resolution is » 30 cm from WorldView-3. However, there exist computational methods which can enhance the resolution from such sensors using techniques successfully applied to date to surveillance and microscopic imagery over many years called super-resolution restoration (SRR).

We have developed a novel super-resolution algorithm/pipeline to be able to restore higher resolution

image from the non-redundant sub-pixel information contained in multiple lower resolution raw remotely

sensed images. As we show in this paper we demonstrate that with a stack of HiRISE images we can achieve

О up to 5 cm resolution from an orbit altitude of 300 km. With 3D information available from the same sensor О

at 25 cm (using shape-from-shading), this now allows us to interpret the surface formation process in a wholly different manner.

The Gotcha-PDE-TV GPT-SRR technique was developed [16] within the EU-FP7 PRoViDE (http://provide-space.eu) project to obtain improved scientific understanding of the Martian surface using a combination of orbital and rover imagery and in future to better support several mission critical engineering rover operations, such as landing site selection, path planning, and optical rover localisation. The technique is unique, since (a) we not only use sub-pixel information from small translational shifts but also restore pixels on an ortho-rectified grid from different (comparably large) viewing angles, and are therefore able to achieve a 2-5x enhancement in resolution; (b) we use a novel segmentation-based approach to restore different features separately; (c) apply a state of the art Gotcha matcher and PDE-TV regularization to provide accurate and robust (noise resistant) restoration. GPT-SRR is applicable whenever there exist sub-pixel differences and there are comparably large view zenith angle differences, which is always the case in orbital images, between multiple image acquisitions even if taken at different times with different solar illumination conditions. Each view is subjected to different atmospheric blurring and scattering but as long as the atmospheric transparency is sufficiently high, Gotcha-PDE-TV SRR can be applied.

This paper will describe the new Gotcha-PDE-TV algorithm and investigate its current performance. The technique will be demonstrated with initial experiments performed using 8 repeat-pass 25 cm HiRISE images covering the MER-A (Mars Exploration Rover - http://mars.nasa.gov/mer) Spirit rover traverse in Gusev crater in order to resolve a 5 cm Super-resolution restoration (SRR) image of the area. This set of superresolution images around the MER-A and MSL track is now being analyzed by colleagues on the MER& MSL science teams in association with the rover imagery in order to try to quantify what additional information on Martian surface processes can be derived given the 5 times higher spatial resolution compared to HiRISE.

66 DOI: 10.7256/2453-8817.2017.2.22876

This resolution is comparable to rover imagery at a stand-off range of 5 m from the rover cameras but in our case this high spatial resolution («5 — 10 cm) can be simulated from many hundreds of kilometers away from the rover traverse. This means that in future, more detailed planetary exploration could be performed from orbit, where there are no autonomous rover vehicles available, by ensuring that there are many repeat images of the same surface area.

2. Context and reviews

SRR techniques attempt to restore higher resolution images from non-redundant information contained in multiple lower resolution images. The basic idea is that each lower resolution image that is captured is a decimated, aliased version of the true scene. SRR is used to retrieve the most probable "true scene" that can be extracted from these lower resolution images. SRR techniques are applicable if there are repeat images taken from (slightly) different positions or viewing angles so that differences in alignment between the camera and the surface exists. Such differences of alignment will introduce additional sub-pixel information of the true scene.

Early work on SRR in computer vision was mainly achieved by exploring the shift and aliasing properties in the frequency domain [18], but these techniques are restricted in the observation/degradation models they can handle. Nowadays, SRR is mostly performed in the spatial domain, mainly for its flexibility to model all the different kinds of image degradations encountered. The naive spatial domain approach is interpolation-restoration, which is a non-iterative forward approach that achieves non-uniform interpolation on pre-registered low-resolution (LR) images. Forward interpolation based approaches do not guarantee optimality of the estimation. Local registration error can easily propagate and will cause gridding artifacts. Unlike the interpolation-based approaches, statistical approaches relate the sub-pixel information stochastically toward optimal reconstruction. SRR image and registration parameters of LR inputs can both be considered as stochastic variables. The inverse process to find out the most probable true scene can be interpreted within a full Bayesian framework. In order to resolve the Bayesian formulation, many works [13, 8, 2] have followed the Maximum Likelihood (ML) estimator and Maximum a Posteriori (MAP) approaches. To resolve ML estimator function requires expensive manipulation of high dimension matrices and therefore a Back-Projection Function (BPF) is normally applied to simplify the large set of sparse linear equations [7]. The ML estimator without regularization is usually very sensitive to noise and registration parameters of LR inputs [2]. Therefore current state of the art SRR techniques follow the MAP approaches, but vary in the observation models and priors. There are three commonly used priors for solving the MAP equation of SRR. The first one is the Gaussian Markov Random Field (GMRF) [5], which takes the likelihood of the prior in the form of a symmetric positive matrix of the derivative operator of LR images, balancing local and spatial smoothness. A common criticism of GMRF is its disadvantage in preserving sharp edges in SRR. The second approach is the Huber MRF (HMRF), which resolves local smoothness whilst preserving sharp edges using the Huber function [13]. The third generic image prior is through Total Variation (TV), which is a commonly used image de-noising technique. TV calculates the total amount of change via a Laplacian operator. More recent works in SRR employ the TV as a regularization prior. [3] introduced bilateral TV (BTV) for reduced computational complexity and improved robustness. [1] proposed an improved regularization method based on the coupling of fourth order Partial Differential Equation (PDE) and a special shock filter to remove the jittering artifacts from TV.

In this paper, we propose a further optimized TV algorithm, called Gotcha-PDE-TV, based on an unique adaptive least-squares correlation (ALSC) matcher called P-Gotcha described in [14] which has been successfully applied to topographic mapping and co-registration of multi-view imagery from HRSC, CTX and HiRISE in [9] http://www.sciencedirect.com/science/article/pii/S0032063315003591 - bib9. This paper will demonstrate optimality of the Gotcha-PDE-TV SRR technique by applying the precise sub-pixel motion prior in MAP reconstruction focusing of orbital images of Mars. For computational reasons, we do not model all the observation parameters such as surface illumination, surface albedo, and camera specifications. Instead

DOI: 10.7256/2453-8817.2017.2.22876 67

for Mars images, we model the precise camera orientation/motion, simplified optical blurring effects, down-sampling effects, and noise.

3. Algorithm and methods

3.1. MAP SRR model

In a generic SRR model (1), where Yk denotes the k-th LR image, X denotes the HR image. Fk, Hk, Dk, and Vk denote the observation parameters, where geometric motion information, optical blurring effect, down-sampling effect, and noise encoded for the k-th LR frame, respectively.

Yk=DkHkFkx+vk, m

where k = l,2,...,K.

Let M denote the observation parameters among LR inputs. The HR image and observation parameters can both be regarded as stochastic variables and hence the SRR model can be interpreted within the full Bayesian framework (2).

X = argmaxPr(X | Y) = argmaxPr(X,M 17) = argmax ———-—-———-. (2) X X x Pr(Z)

Since X and M are independent:

Л

rt S

н

ta

x = argmaxP£Щв. (3)

S

o

Pr(X) is the prior term on the desired HR image and Pr(M) is a prior term on the geometric motion vector. Because any pixel value Xij in the reconstruction is highly correlated with their neighbours, we assume Pr(F | X,M} is normally distributed. The probability of the observed pixel value in the LR image is given

<! in (4) with a zero mean and standard deviation o.

H

o

* Pr(Z|X,M)ooe-1/2a2WI2. (4)

At the current stage, we only deal with a single-sensor (e.g. HiRISE) with LR images captured over a period of time. The relative motion for each pixel is calculated to sub-pixel accuracy with respect to an orthorecti-fied image (ORI). Therefore we assume M is given/calculated beforehand, (4) can be simplified to (5), where A(X) is the regularization cost represented by a non-negative potential function used to define Pr(X) and X is a regularization parameter for absorbing the variance of the similarity and regularization costs.

X = arg max Pr(71X, M) Pr(^) = arg min |Y - MXf + XA(X) } (5)

In TV regularization, A(X) is measured by the l1 norm of the magnitude of the gradient to preserve edges and corners while encouraging local smoothness. Where in (6), V is a gradient operator that can be approximated by Laplacian operators.

4r(X)Hlvxlli- (6)

Merging (5); (6), the MAP equation can be represented as (7).

X = arg min |Y - MXf + X |VX\[ }

(7)

By applying a 4th order PDE (8) to resolve the minimization problem, the staircase effect in TV can be minimised.

|v2x|r IIVJJT

VI >.

(8)

where |i>0, such that:

||VX|| * ||V, jr|| = (x2x + X2y + n2)1/2 = (¡VAf + II2)12 ■

In order to prevent the denominator of PDE, i.e. VAT , approaching zero.

(9)

3.2. Reconstruction algorithm

To resolve the MAP equation from Eq. (7) requires the "true" value for observation parameter, M. In our method, we use a sub-pixel motion map to describe the geometrical correlation for each individual pixel between LR images and a reference ORI for the same scene so that each LR pixel can be fitted onto an interpolated HR grid (for desired resolution scale factor L) as a starting estimation for the steepest descent iteration of the minimisation problem of Eq. (8).

In order to get the motion prior close enough to the true value which is essential for the MAP solution, we use a progressively weighted ALSC/region growing algorithm to produce sub-pixel 2-channel (dx and dy) projection maps for each LR image with respect to the reference ORI. Initial tie-points (TPs) are determined using feature points derived from Scaled Invariant Feature Transform (SIFT). General feature based registration methods assume that image features detected independently on each image are always correct. The repeatability of the detection would be deteriorated when a significant distortion is involved in a matching process. Slight mismatches could have a large impact on constructing the initial High Resolution (HR) grid. Therefore, we developed a Mutual Shape Adapted SIFT (MSA-SIFT) algorithm that uses forward and backward ALSC to iteratively search for a correct TP by adjusting the shape of the correlation matching window as shown in Eq. (10), where Xi is an ALSC searching window starting from the origin of the initial TPs. The window can be translated and/or skewed as represented by A.

TP

11 MSA-SIFT

= arg minJjLY/

X

The algorithm of MSA-SIFT consists of the following:

LR

AX,

ORI

(10)

(i) Detection of a scale invariant feature and its scale.

(ii) Iterative update of a circular scale invariant region to an elliptical region using a second moment matrix.

(iii) Initial normalisation using the result from ii.

(iv) Refinement of the result using forward and backward ALSC on both images.

(v) Going back to (iv) until it converges (optional).

(vi) Go back to (ii) until convergence (optional).

TPMSA_SIFT have sub-pixel accuracy and are considered as seed points to iteratively adjust transformation and move to the neighbouring points in the P-Gotcha algorithm as described in Eq. (9). Subjective constraints,

л

3

a

rt s н

на Я

О

о о

w г м ее

О

г

©

e.g. quality, error, are determined to obtain a motion map for each LR image with respect to an ORI. Each pixel value (dx, dy) in the motion map represents the x andy vector pointing to the same interpolated ORI grid with scale factor L. Other observation parameters (Hk, Dk, and Vk) for a LR image are projected by the geometric motion parameter (Fk) from the motion map to find the minimum squared error (MSE).

In a steepest descent approach for the minimization problem we use Eq. (12) to resolve the PDE decomposed MAP Eq. (11).

dx_ 8t

= -$FTHTDT (FHDX - 7)+ V2

У2Х 1 У2Х

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

У2ЦХ

(11)

-ЭZkFkTHTkDTkDTk (FkHkDkXn-Yk)+V2

V X.

y2x

Ц П

v2x.

(12)

•n я

Ol

a

rt s н

на Я

О

о

О W

г м ее

£ О

г

©

where y is the step size in the direction of the gradient, FT, HT, and DT is the transpose of the projection vector, Point Spread Function (PSF) that is assumed to be a small Gaussian kernel with standard deviation o to be 1, and a down-sampling operator, respectively for the k-th LR image.

Most MAP SRR approaches in computer vision assume a simple projection motion prior to a LR image sequence. However, this is not the case in planetary orbital image datasets where motion vectors can vary dramatically due to camera viewing angle differences. We use a novel back projection scheme that reconstructs different areas (S) from the LR images separately with respect to the segmentation from the tiled motion vectors with different levels of pixel distance (r) when compared with the reference ORI, such that:

1=1

k H^max

-D . <Г.

mm Hi

(13)

(14)

where D is the distance of the motion vectors within an area S, such that rS is less than a threshold, T. Each LR measurement Yk in Eq. (12) within an area Si will be compared to the degraded estimation of HR frame Xn in the n -th steepest descent iteration separately. In such an approach, any area with more features (small rocks, edges) will be divided into more reconstruction tiles in order to preserve the features, whilst flat featureless areas will be reconstructed jointly to reduce noise and speckle effect from a PDE. In other words, neighbouring correlations have less effect on the fine detail of features but have more effect on flat featureless areas. In addition, with this tiled back projection scheme, the neighbouring pixels outside the defined area (tile) will not contribute to the SRR within the defined area (tile) according to the normal distribution assumption of Pr(Y|X,M) in Eq. (4). For flat areas where LR pixels are over-determined, a PSF with larger o is used. For highly featured areas where LR pixels are under-determined, a PSF with a smaller o is used.

3.3. Method

The current implementation of the proposed Gotcha-PDE-TV SRR algorithm is shown schematically in Fig. 1. We take roughly aligned overlapping LR images (K) and an ORI (if available) as input to estimate the HR image with a given scaling factor (L).

Fig. 1. Flow diagram of the proposed Gotcha-PDE-TV processing chain.

The processing pipeline (see Fig. 1) starts with a scale invariant feature detection and matching process to predict TPs for each LR images with respect to the reference frame. Then an iterative MSA method is performed based on the initial TPs to further improve the sub-pixel accuracy followed by a RANSAC process for outlier removal. These optimised TPs are then used with a pyramidal version of Gotcha as seed points for an ALSC/region growing process until most pixels in the LR images find their optimal sub-pixel correspondence with respect to the reference frame. These sub-pixel correspondences are collected to form (K) 2-channel motion maps with sub-pixel x andy translation vector encoded. LR pixels, which do not match with any position in the reference grid, will be removed from the K-dimensional LR matrix. If a position in the HR grid does not have any corresponding motion vector from all (K) motion maps, this HR pixel will be propagated by its neighbouring HR pixels.

The motion maps provide the initial degradation information of F in the similarity measurement term of the MAP estimation. LR images and the reference ORI are resized by the defined scaling factor L and are segmented to (S) tiles according to a given threshold (T) of the maximum difference of the magnitude of the distance of the motion vectors.

The next step is resolving the MAP equation using the following method:

л

3

i

a

rt s н

на Я

О

о о

w г м ее

О

г

©

(i) For the same area, each tile (r) of each LR image (k) is projected with motion vector (F), convolved with PSF (H) which is assumed to be a small Gaussian kernel with various standard deviation (ct) according to the size of segments (S), down-sampled (D) with the defined scaling factor (L) and compared with its estimated HR image tile sequentially.

(ii) Go back to (i) for the next image (k) until all images converge.

(iii) Add the transposed difference vector (FT, HT, DT) for the image tile (k, r).

(iv) Add the smoothness term and decompose the TV regularization term with the 4th order PDE.

(v) Go back to (i) for the next steepest descent iteration until it converges.

(vi) Collect the HR result for this tile (rz) and go back to (i) for the next tile (r ) until all segments (S) converge.

(vii) Collect the results for all HR segments (S) and reconstruct the full HR grid.

(viii) Finally a series of post-processing is performed based on the HR reconstruction including noise filtering and deblurring.

This implementation includes several tiling and pyramidal approaches in order to decrease the processing time. The potential of parallel processing for the ALSC/region growing, tiled back projection and regularization is indicated in the above processing chain (see Fig. 1). A porting of the current implementation to a network of high speed Graphics Processing Unit (GPU) processor should be feasible as there are existing implementations [11, 6, 12] for most of the core parts, i.e. seeded Region Growing, SIFT, TV) to port onto a GPU. If we are able to achieve order(s) of magnitude increase from parallel processing in throughput then we will eventually be able to process full HiRISE scenes. Consequently, our ability will be enhanced to study very fine-scale processes from Martian surfaces such as gulley and RSL formation and address much better the question as to whether these originate with liquid water. Currently, HiRISE has acquired, up to the end of Mars Year 31, around 400 areas (0.02% of the « 145 Msq. km). Martian surface area assuming a typical HiRISE scene size of 6x12 km) with four or more repeat coverages [15].

н

4. Experimental results

na EC

O

o

Initial experiments have been performed using the elaborated Gotcha-PDE-TV algorithm for 8 repeat-pass

ijj

25 cm HiRISE images listed in Table 1 covering the MER-A Spirit rover traverse in Gusev Crater to resolve a 5 cm SR image of the area as shown in Fig. 2 and the zoomed-in view for randomly picked places from Sol 524 to Sol 580 in Fig. 14 in comparison with 25 cm HiRISE image for the same area shown in Fig. 13.

£

©

Table 1

LR inputs of 8 repeat-pass HiRISE images

ID Acquisition Date

LR1 ESP-011943-1650 12 February 2009

LR2 ESP-016677-1650 15 February 2010

LR3 ESP-019301-1650 8 September 2010

LR4 ESP-025393-1650 27 December 2011

LR5 ESP-025815-1655 29 January 2012

LR6 PSP-001513-1655** 22 November 2006

LR7 PSP-001777-1650* 12 December 2006

LR8 PSP-010097-1655 21 September 2008

-23270.2« -8f 33673S6 Mrttii

Fig. 2. MER-A SRR mosaic covering the whole rover traverse shown in ArcGIS®.

Owing to the current very lengthy computation times of each SRR image tile (12-24 h depending on different processing parameters, for a 2048x1024 tile with 8 input LR images running on a 16 core, 8 GB RAM computer), it is not yet feasible to apply SRR to a full HiRISE image. Therefore, a set of smaller image tiles has been processed along the MER-A rover track and their coverage and the corresponding tiles are shown in Fig. 2.

The reference ORI for the SRR is created using the left image of the stereo pair PSP-001513-1655 ,, and PSP-001777-1650 , described in [17] http://www.sciencedirect.com/science/article/pii/S00320633150033591 - bib18. A comparison between one of the LR images and HR images can be seen side by side for 3 randomly picked places (A, B, C) within the Homeplate area SRR view in Fig. 3 and are shown individually in Fig. 4, Fig. 5; Fig. 6.

Fig. 4 shows that the proposed SRR algorithm is able to bring out individual rocks (size < 75 cm), which are not clear or unrecognizable in the original HiRISE image. This is essential for rock detection/classification and examining surface rock distribution for understanding the surface roughness. From a better understanding of the rock distribution, an optimal path planning can be calculated to better support the engineering teams of future surface missions. For the most recent rover, [4] showed down-selection of putative landing sites had to meet the criterion that rock height of less than 0.5% probability of at least one <0.55 m high rock in a 4 m2 area, equivalent to a rock abundance of <8%.

з

i

a

rt s н

на Я

О

о о

w г м ее

2 О

г

©

Fig. 3. A portion of the 5 cm MER-A SRR image mosaic around the Homeplate area, the most south-east tile shown in Fig. 2, showing locations of the 3 selected areas shown at higher resolution in Fig. 4, Fig. 5; Fig. 6.

I

Fig. 4. Comparison between 25 cm HiRlSE ORI image (left) and 5 cm SRR image (right) for area (A).

Fig. 5 shows a portion of SRR of mixed features including rocks as small as 10-50 cm, dunes, and hill slopes. This provides important knowledge on surface morphology and deposition study. Such enhanced structural/linear features will also improve ground-to-orbit data fusion, i.e. Navcam to HiRISE co-registration as described in [17], which will significantly improve an optimal rover localisation.

Fig. 6 shows the optimality of the SRR algorithm in preserving sharp edges. The restoration of sharper edges is important for studying sedimentary deposition and surface change monitoring.

Furthermore, we are able to enhance and composite rover tracks that appeared in different HiRISE images by using different parameters for each LR image depending on the different desired area, as shown in Fig. 7. In comparison of the enhanced and composited rover track with the rover imagery shown in Fig. 8, we are able to perform high accuracy rover localisation as well as validate the spatial resolution, in this case by measuring the outer-wheel and inner-wheel spacing shown in Fig. 9. The maximum difference between the rover track outer spacing from SRR image and Navcam orthorectified mosaic is within 8 cm (1.6 pixels in the SRR image), which is subject to Navcam orthorectification distortion and possible Martian surface change over a 5 year long period.

In addition, SRR imagery is applicable to improve knowledge of rock size distributions, which is critical for understanding the geological and geomorphic history of a surface [4] as well as the potential navigability of the surface. Fig. 10 shows that in 25 cm HiRISE images, rocks less than 150 cm diameter are barely visible and are hard to detect, whereas in 5 cm SRR, rocks larger than 30 cm diameter are fully resolved shown in Fig. 12.

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

л

3

a

rt s н

на Я

О

о о

w г м ее

2 О

г

©

Fig. 5. Comparison between 25 cm HiRISE ORI image (left) and 5 cm SRR image (right) for area (B).

Fig. 7. Comparison between 25 cm HiRlSE ORl image (left) and 5 cm SRR image (right) showing composited enhanced rover tracks.

Fig. 8. Comparison between rover track compositing in SRR image andJPL vertical projected Navcam RDR (2 nnd95 ilfawvrtz0p1725l000 m2 and 2 nnd54 ilfavvrtqwp1605l000 m2) in Homeplate area.

For an area within the HiRISE image and a corresponding SRR image around an impact crater near the MER-A traverse Sol 150 and 151, rocks are automatically detected based on the Mean-shift segmentation and Support Vector Machine (SVM) classifier [16]. For rocks with diameters larger than 150 cm there are 22 detected from the original HiRISE image and only 1 rock detected with a diameter between 50 cm and 150 cm. On the other hand, in the SRR image, Fig. 12 shows that there were 33 rocks with diameters larger than 150 cm, 111 rocks with diameters between 50 cm and 150 cm, and 9 rocks with diameters between 30 cm and 50 cm. We have also compared the rock detection results, shown in Fig. 11, on the enlarged (bilinear interpolation) and high-pass filtered HiRISE image. Although some rocks with diameters larger

s

о

U -Я

3 0|

а

<s s н

S=

о о о

W

г

M ее

о

г

©

Fig. 9. Comparison between specially enhanced rover track composition in SRR image and orthorectified rover Navcam mosaic in Homeplate area.

Fig. 10. Automatically detected rocks (labeledgreen) of 25 cm HiRISE image (PSP-001513-1655) with 20 pixel grid (5 m) around an impact crater close to MER-A traverse at ~ (175.51045 deg, -14.58461

.a S

a

rt s

и:

than 150 cm were able to be individually detected (viz. they were detected as a single rock in the original HiRISE image) and also some blurred rocks with diameters between 50 cm and 150 cm have been enhanced, there were still a very large number of rocks which cannot be resolved by simple high-pass filtering. Table 2 summarises these results showing that there are large numbers of rocks, which are not clear enough for automated detection/classification or manual recognition in the original HiRISE image. However, with the SRR, a much greater number of rocks can be detected and therefore provide stronger evidence to support an application such as the selection of a future landing site.

Table 2

Й Я

О

о о

w г

M ее

О

г

©

Accumulated number of rocks in HiRISE and SRR image

Diameter of rocks Num of rocks (HiRISE) Num of rocks (filtered) Num of rocks (SRR)

£>>150 cm 22 25 33

D > 50 cm 23 31 144

D> 30 cm 23 31 153

More SRR experiments and processing have also been performed (not shown here) for MER-B Victoria Crater, Endurance Crater, Santa-Maria Crater and the entire MSL rover traverse to die. Some of the SRR results have been integrated into an interactive Web-GIS system developed by partners at the University of Nottingham within the PRoViDE project, called PRoGIS7, for visualisation in a multi-resolution co-registered context using SRR image, HiRISE, CTX and HRSC which is designed to serve public outreach and educational

3

|

а

Fig. 11. Automatically detected rocks (labeled green) of high-pass filtered HiRISE image (PSP-001513-1655) with 20 pixel grid (1 m) around the same impact crater close to MER-A traverse at ~ (175.51045 deg, -14.58461 deg).

rt S H

M Я

О О О

w г

M ее

О

г

©

Fig. 12. Automatically detected rocks (labeled green) of 5 cm SRR image with 20 pixel grid (1 m) around the same impact crater close to MER-A traverse at ~ (175.51045 deg, - 14.58461

purposes [10]. The greatest single limitation to the existing technique is the slow computational speed, in addition to ensuring that there are sufficient repeat images (5-10) of sufficient clarity/atmospheric transparency.

Fig. 13. A portion of 25 cm MER-A HiRISE image with rover traverse from Sol 524 to Sol 580.

5. Conclusions and future work

Any planetary geologist or geo-morphologist is likely to have a strong interest in exploiting the highest possible resolution 3D image dataset. SRR will assist them greatly in formulating and testing hypotheses about planetary surface processes, as they will be able to apply their knowledge and understanding based on their terrestrial fieldwork. The high spatial resolution imaging data is an active driver for many applications, such as studying surface processes, which are not visible or not clear enough via known low-resolution data. Geologists can achieve more reliable classification and inference from super-resolution restored features such as rocks, sedimentary layers, cliff cross-cutting profile, etc.

л

3

i

a

rt s н

на Я

О

о о

w г м ее

2 О

г

©

Fig. 14. A portion of 5 cm MER-A SRR image mosaic from 8 repeat-pass HiRISE images showing the same place from Sol 524 to Sol 580.

This paper describes our novel SRR algorithm, called Gotcha-PDE-TV (GPT-SRR), to address the reconstruction of fine-scale details from multi-frame repeat-pass orbital imagery. We show use of an innovative tiled MAP approach to restore different feature from LR images. We exploit the accurate sub-pixel motion estimation using Gotcha and robust PDE based TV regularization process. The technique has been demonstrated here with experiments on 8 overlapping 25 cm MRO HiRISE images covering the MER-A Spirit rover traverse to resolve 5 times higher spatial resolution. We do not yet know, and are not able to test, owing to the huge computational time, how many images yield what resolution but determined this heuristically at the 5 cm shown for the 8 input 25 cm images.

Gotcha-PDE-TV SRR is applicable whenever there exist sub-pixel differences, which is always likely to be the case for repeat orbital images, even if taken at different times with different viewing (zenith) and solar illumination (azimuth) conditions. We are not able to test this with different solar zenith as MRO is in a fixed repeat orbit. Each view is subject to different atmospheric blurring and scattering but as long as the atmospheric transparency is sufficiently high (clear), Gotcha-PDE-TV GPT-SRR can be applied.

We aim to process all available image datasets in future where we have repeated multi-view imagery starting with HiRISE first and then apply these techniques to HRSC, CTX, HiRISE, THEMIS, MOC and Viking Orbiter into geo-referenced SR mapped datasets after the proposed GPU porting. We also plan to apply such techniques to the retrieval of 3D heights where we have multiple stereo-pairs available. These geo-referenced SRR datasets will greatly support the geological and morphological analysis and monitoring of Martian surface processes especially change detection features in future planetary research. They can also be applied to landing site selection to spot surfaces which may cause difficulties for any future rover as well as provide a much better dataset for subsequent geological and geomorphological analysis.

We believe that the technology developed here has huge potential, not only to other Solar System solid earth targets but also to the design of future missions, which will still be severely limited by telecommunications bandwidth but also by light travel time. Transmitting back long video sequences of LR imagery, which could then be employed for SRR, could result in substantially higher scientific returns from orbital missions.

pa

It may also be applied to space telescopic images of objects outside our Solar System such as exoplanets.

О о

Статья впервые опубликована: Planetary andSpaceScience, 2016, № 21, pp.103-114

м ffl

2

Библиография

1. Bouzari H. An improved regularization method for artifact rejection in image super-resolution // Signal Image Video Process., 6 (2012), pp. 125-140.

2. Capel D. Image Mosaicing and Super-resolution, 2004, Springer-Verlag, London Ltd., eBook, ISBN 978-085729-384-8 (Springer-Verlag, London, Berlin, Heidelberg).

3. Farsiu S., Robinson D., Elad M., Milanfar P. Fast and robust multi-frame super-resolution // IEEE Trans. Image Process., 13 (10) (2004), pp. 1327-1344.

4. Golombek M., Grant J., Kipp D., Vasavada A., Kirk R., Fergason R., et al.Selection of the mars science laboratory landing site // Space Sci. Rev., 170 (1-4) (2012), pp. 641-737.

5. Herman G.T., Hurwitz H., Lent A., Lung H.-P. On the Bayesian approach to image reconstruction // Inf. Control, 42 (1) (1979), pp. 60-71.

6. Heymann S., Frhlich B., Medien F., Mller K., Wiegand T., 2007. Sift implementation and optimization for general-purpose gpu. In: Proceedings of WSCG 07.

7. Irani M., Peleg S. Motion analysis for image enhancement: resolution, occlusion and transparency // J. Vis. Commun. Image Represent., 4 (4) (1993), pp. 324-335.

8. Kaltenbacher, E., Hardie, R.C. High resolution infrared image reconstruction using multiple low resolution aliased frames // IEEE National Aerospace Electronics Conference, 1996, 2, pp. 702-709.

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

9. Kim J.-R., Muller J.-P. Multi-resolution topographic data extraction from martian stereo imagery // Planet. Space Sci., 57 (2009), pp. 2095-2112.

10. Morley J., Sprinks J., Muller J.-P., Tao,Y., Paar G., Huber B., Bauer A., Willner K., Traxler C., Garov A., Karachevtseva I. Contextualising and analysing planetary rover image products through the web-based progis // Proceedings of Geophysical Research Abstracts (EGU), 2014, vol. 16.

11. Park S., Lee J., Lee H., Shin J., Seo J., Lee K.H., Shin Y.-G., Kim B.Parallelized seeded region growing using cuda // Comput. Math. Methods Med., 2014 (2014), p. 10.

12. Pock T., Unger M., Cremers D., Bischof H. Fast and exact solution of total variation models on the gpu // CVPR Workshop on Visual Computer Vision on GPUs, 2008.

13. Schultz R.R., Stevenson R.L. A Bayesian approach to image expansion for improved definition // IEEE Trans. Image Process., 3 (3) (1994), pp. 233-242.

14. Shin D., Muller J.-P.Progressively weighted affine adaptive correlation matching for quasi-dense 3d reconstruction // Pattern Recognit., 45 (10) (2012), pp. 3795-3809.

15. Sidiropoulos P., Muller J.-P. On the status of orbital high-resolution repeat imaging of mars for the observation of dynamic surface processes // Planet. Space Sci. (2015).

16. Tao Y., Muller J.-P. Automated science target selection for future mars rovers: a machine vision approach for the future esa exomars 2018 rover mission. In: Proceedings of Geophysical Research Abstracts (EGU), 2013, vol. 15.

17. Tao Y., Muller J.-P., Willner K., Morley J., Sprinks J., Traxler C., Paar G. 3d data products and web-gis for mars rover missions for seamless visualisation from orbit to ground-level. In: Proceedings of ISPRS Commission IV Symposium, 2014, vol. 8.

18. Tsai R.Y., Huang T.S. Multiple frame image restoration and registration // Adv. Comput. Vis. Image Process. (1984), pp. 317-339.

M К

References (transliterated)

1. Bouzari H. An improved regularization method for artifact rejection in image super-resolution // Signal Image Video Process., 6 (2012), pp. 125-140.

2. Capel D. Image Mosaicing and Super-resolution, 2004, Springer-Verlag, London Ltd., eBook, ISBN 978-0-

W

85729-384-8 (Springer-Verlag, London, Berlin, Heidelberg).

3. Farsiu S., Robinson D., Elad M., Milanfar P. Fast and robust multi-frame super-resolution // IEEE Trans. Image Process., 13 (10) (2004), pp. 1327-1344.

4. Golombek M., Grant J., Kipp D., Vasavada A., Kirk R., Fergason R., et al.Selection of the mars science labo-

©

ratory landing site // Space Sci. Rev., 170 (1-4) (2012), pp. 641-737.