The compression algorithm of hyperspectral aerospace images with use of mathematical processing and intrabands correlation Текст научной статьи по специальности «Компьютерные и информационные науки»

Journal of Siberian Federal University. Engineering & Technologies, 2018, 11(8), 882-891

yflK 004.627

The Compression Algorithm of Hyperspectral Aerospace Images with Use of Mathematical Processing and Intrabands Correlation

Assiya Zh. Sarinova and Alexander V. Zamyatin*

National Research Tomsk State University 36 Lenin, Tomsk, 634050, Russia

Received 15.12.2017, received in revised form 29.07.2018, accepted 16.10.2018

This article reveals the features of the channels of hyperspectral aerospace images. There has been suggested the algorithm with interchannel finding of the best channel of intrabands correlation and the use of indexed lossless encoding, which allows to reduce the dimensions of the image channels and to convert them before compression. Also the article shows the results of experiments in comparison with universal archivers such as Winrar, 7Z, and JPEG Lossless according to the compression degree and several parameters: the interchannel correlation, geometric image size, number of channels, number of channel groups. Furthermore, the author of the article studies the options of compression with the search of the main channel of the correlation and without it; these options demonstrate the increase in compression degree due to differential transformations, which allow to store the information in a more compressed form.

Keywords: hyperspectral aerospace images, the compression algorithm, intraband correlation, indexed encoding, channel groups.

Citation: Sarinova A.Zh., Zamyatin A.V. The compression algorithm of hyperspectral aerospace images with use ofmathematical processing and intrabands correlation, J. Sib. Fed. Univ. Eng. technol., 2018, 11(8), 882-891. DOI: 10.17516/1999-494X-0110.

© Siberian Federal University. All rights reserved

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). Corresponding author E-mail address: assiya_prog@mail.ru, zamyatin@mail.tsu.ru

Алгоритм сжатия гиперспектральных аэрокосмических изображений с использованием математической обработки и учетом междиапазонной корреляции

А.Ж. Саринова, А.В. Замятин

Национальный исследовательский Томский государственный университет Россия, 634050, Томск, пр. Ленина, 36

В данной статье исследованы особенности каналов гиперспектрального аэрокосмического изображения. Предложен алгоритм с межканальным нахождением наилучшего канала междиапазонной корреляции и применением индексированного кодирования без потерь, позволяющий уменьшить размеры каналов изображения и преобразовать их перед сжатием. Приведены результаты экспериментов в сравнении с универсальными архиваторами Winrar, 7Z и JPEG Lossless по степени сжатия и нескольким параметрам: учету межканальной корреляции, геометрическому размеру изображения, количеству каналов, количеству групп каналов. Рассмотрены варианты сжатия с поиском главного канала корреляции и без него, демонстрирующие повышение степени сжатия за счет разностных преобразований, позволяющие хранить информацию в более уменьшенном виде.

Ключевые слова: гиперспектральные аэрокосмические изображения, алгоритм сжатия, междиапазонная корреляция, индексированное кодирование, группы каналов.

Introduction

Modern satellite-based centers of space monitoring and remote sensing of the Earth (RSE) efficiently accept, register, process, archive and distribute large amounts of data, which are sometimes of hundreds of gigabytes.

It is known that the aerospace images (AI), RSE are characterized by different features: spectral, radiometric, spatial resolutions, geometric sizes of the board, which have different number of channels and ranges.

In the field of compression of hyperspectral AI we already have interesting research results [113], however they have some disadvantages. For example, in the works [2, 3] there were presented the approaches to lossless compression of hyperspectral AI using an adaptive prediction and back-search schemes. The proposed scheme is based on the approach of the table prediction and uses two new approaches to improve the compression performance. The first approach uses the spatial correlation of the data and formulates a conclusion of spectral prediction in the filtering process of Vinner. In the second approach, the search scheme is used instead of reference tables, that significantly reduces the storage requirements. The search is greatly reduced using the quantification index. The disadvantage of these approaches is a partial loss of information due to quantification phase.

Some researchers have considered the characteristic features of hyperspectral AI. So, the technique of lossless compression using the spatial-spectral encoding is researched in the following works [1, 4, 7]. The researchers have proposed a new hybrid context prediction for the lossless compression of hyperspectral data, also the spectral correlation matrix is calculated. The authors suggested a new

technology of finding a high correlation of channels groups of hyperspectral data cube on the basis of the group of squared correlation. The disadvantage of such approaches is the time required to compress and restore the image, which depends on calculation of spectral correlation matrix, and also the proposed schemes produce an average compression ratio which reaches only 3,2.

In the article [5, 9] it is proposed to use randomized methods for reducing dimensionality for effective coverage of the correlation structure and encoding the part remained, and for ensuring lossless compression. This approach is used especially for the efficient lossless on-board compression, data transmission and reconstruction of HSI data. One of the problems for the residual encoding is the speed. In this regard, a recent encoding development showed a throughput which reached 75 GB/s on the GPU. In the works [6, 8, 10] there were resolved two important issues related to the storage and transmission of hyperspectral images, considering the lossless compression models. Also there was studied the SLSQ algorithm and its condition in modern lossless compression schemes, and this algorithm is suitable for on-board implementation. Also two approaches to improve the SLSQ compression are considered. The first approach is a pre-processing of the scheme which orders the channels before compressing a hyperspectral image, also we studied the correlation between the channels. The second approach is based on the pre-processing time and this individualizes a set of channels which are better compressed with two-dimensional inter-channel predictor instead of three-dimensional SLSQ. The main idea of the approach is determined by the fact that the SLSQ NR-groups of the algorithm are not effective, reaching a compression degree of 3,3.

Approaches of lossless compression are considered by most researchers, because the lossless algorithms are effective as the information is important in many fields such as RSE environmental protection, the earth's surface monitoring, military purposes, etc.

Based on the conducted studies and the above approaches to algorithms and methods of compression of hyperspectral AI it can be assumed that these disadvantages stimulate intensive researches aimed at finding new effective solutions which provide high rates of compression degrees and calculators for processing hyperspectral AI, as well as the development (modification) of new algorithmic software.

One of the perspective ways of solving the problems of hyperspectral AI compression is the development of algorithmic software with the new methodology of accounting an inter-channel correlation and creating new algorithms. There are some works devoted to this field [11-13], which suggest the need to develop new algorithms and improve existing ones in lossless compression for hyperspectral AI. There are also the developments in the works [14, 15], where preparatory lossy and lossless processing of algorithms is proposed. In this regard, this article proposes some modifications and new stages of the algorithm of lossless compression based on the conducted researches.

Stages of the compression algorithm

There has been suggested the algorithm with interchannel finding of the best channel of intrabands correlation and the use of indexed lossless encoding, which allows to reduce the dimensions of the image channels and to convert them before compression. As a result, the compression degree of the channels of hyperspectral AI processed by well-known statistical algorithms has become much higher. Also some results of study of the compression degree in the number of channel groups were revealed.

Set of the algorithm stages:

1) the calculation of the correlation matrix between AI channels, finding the best correlated channel pairs in all possible combinations, the subtraction of the best pairs of AI channels;

2) the indexed encoding is the encoding in which the range of hyperspectral data ranges from 0 to 255 bytes, this proposed algorithm reduces the range which varies from 0 to 68. The index is the number of the position of the data range value of additive RGB color model. The effectiveness of indexing and increasing the degree of compression is that the indices are in a small range, so they are well-compressed by entropy encoding;

3) the statistical compression algorithm.

Stagesofthcalgorithm.

Stage O.CogsideroOionofintnabands con-elation.

Calculating Ihc correMrop vfluh (introbandlgeooggeneai between Syperspex)rai An chamicls allows to redoce nhe datante. so fewer numger ml rates for their ntorage is needed. Tic lef correlntedcnannet shanld sefouno. Then meconsideration ofintrabands sorrelation between the serened chamiete a° llyderfee ctral aeroeoeee mane gy snlectigg tee corretation vahie andOoomingtho deviations (driferences) oftye origlnai rlatoshouMbe tckc edOo^l^(^um.

Foti betailed description of the first stage it is necessary to consider the following objects:

- the originaHmage- matrtxof theimagovalues \\m,s,kj, wnere m,n,I be thelndieesoC iin^s, cohlmnsesdchennels odnde oaigmal image, m= r,a,...,M, a= 1,a,...,d, k = r,a,..],C;

- alC ^ —earray for vCuessterage wher^ caienlatingthe matbematioal egpcot:asion of otsi clannel;

h Y\m,n, Oi htbe array eaoptahlngdiíferences (deviations) values between the value of the mathematieaiexpectetion Af] chd \lm,n ,k];

- Cl— - the array foostorade fesoercentien vfluesrsr bd^^i^en tiie agjeceat rangeslcllonneij);

a Kt°t l s die gr^st c;rrelated channel;

- I"[p,r,/:]- tte errao for plgclnggitfercnces(deviotio nf value s bctwee n fhechannels I'|mm,kf anH pn,n^k(;

Step! Tn ^111^6 the mojnomaticU onpeclation afsach channeloetllo original image -la,a,kr and plasetiie nahies m tlte oh.1 so[r[[.

Step 2. fp sn arch jhccdanneisI[п:In,k]ln all o.&+T[ n^noph^^ 11^ ^he aria, Kxh

stc/l ti calcniajolne correistion eor lon tdebasa aIA[k] ondI[g,«,a])fot ea^eh ^t^érftom Cl sneiia1)]le K chrnnels of the originae image -lm,a,kr. The result should be placed in Clk], knr,a,...,K-r.

(-i (-' [m,a,k]-A[k] )x (- rm,a,k-l]-A[k-1])

Clk r = -

[m,n,k]-A[k] )2x£ (I' [ m,n,k-1]-A[k-1J)2

m,x m,x

Step 4. To calculate the differences (on the basis of A[k], Kbest and I[m,n,k]) between the actual values ff \[m,n,k\ and the mathematical expectotionfor eaph paiir fram v^l ava^Hable K

and KA„, channels of the original image. The result should be placed in \'\m.n.k\. k=\.2.....K-\.

V [/n, 71, k] = £ (l[m, n, k] - A[£]).

m,n

Step 5. To calculate the difference (on the basis of A[k] and C[k]) between the elements I'[m,n,k] andthecorresponding valuesof theoriginalimage I[m,x,k]in each from K channels

„ where СШ >0, Г\m,n,k -I\т,п,к-П Y[m,n,k] = \ , ' rL ;iJ L . for m = 1. Мл = \.N The result should be

[where С[Аг] < 0,1[т,л,к]

ptececl in Il'r,iyг,^7^iel.

As s 1r<agг;^:^t 1;]:ю anac C"^^^ basecn on enhintrogands со—еМ-оп valoesC[H] -iformed. aihy snbtrac-cod CdiCfreradaУ conversion) gives a specieX advantage in the compression of hyperspectra] aery opaceimages, and also takes into account such spectral characteristics as high interchannel correlation, that increases the compression degree comparing with the universal algorithms whkh donuCs ons[de r AI faseuxes. Stgnf О.ТО indiced I-iAcaing.

The gfficiency op Cds olgpdishm is time rvrnge gf hynarspecCral ufota varied in a certamrangeof posaigih vaCd usiag add\Cichyl one-dlmnnsronnl dnta cira]nureuIudatasCrueIurar theddues are kept ln ШеСогт ofindiceo.InXea isSPeposstlonnumber of the data range value of hyperspectral AI channel. don, noh-repeftine ivdicea oflhevaluauonaubШaho suta ^ио-игев one kepu inaCena ofHie original values of hyperspectral aerospace images.

Ahefs rmation o.the raw arigmnldnSoatructure by the subtraction of the sequence unit of the vslues tot neceive a OimtniIihylmyge lhus t-ternft ЫосОек mf nts о"Йю1 magechacnelaredefined, ^luis cdeaShr ihree nioih dota slaga-u—s.

LsI ha consider "flee aroyosed algorithm witii "he he\p if Shesbove sdaae.Wh icIraduee tie fotlawmg о^е^ай] cansideration:

- auxiOiarn data sOructures (ADS) to obtain vne-Cime nskmalarrass of e ach image channel I[o,e,H] a Ij[o,«,H], I[o,i,H] a Ie[o,«,H], I[o,i,H] = I3[o,i,H], where 1-3 are ADS numbers;

- ВД in Шеаггау to jsSate thu иощие i^dscet nalues of t,^is pixeSs for eac h sham^l I[o,i,H] and ADS;

- OTigingl indsc^s Cspia^r 1-[о0]Я,Г:], I2[m,n,kS] Il(idr

Sten 1. ConphrthagedSfgctoal uCs hmah=ei ^^^a^E^S l\m,nO\ = I^.o,^, ,T]= Ie[o,i,H], I[o,i,H] a I3[o,i,H] corresponding to the values of the original image I[o,i,H] in the blocks. The result should beput inthe array i r[cs,nJeS.

Step 2. Form the arrays Г[от,и,А-] based on the auxiliary structures by replacing for each Ij [nun,A],

I j [»7,«, A"]

I2[w/,w,A], \3\m.n.k\. The result is put into the arrays: I"[ot, w. A]: \"\m,n,k \ = < 121т.п.к \.

I3[o, i, H ]

Step 3. Form one-dimensional arrays of the original indices values I[A] on the basis of arrays

T'^^el!

1"[»;,;?,А] and ADS. The result is put inI[A]: I[A]:I[Ar] = \ \"\m.n.k\ e I2.

\"[m,n,k] e I3

In the result, there has been formed the array I[A] considering the formed array V[m,n,k\ and intrabands correlation C[H].

The advantage of the proposed algorithm is that the indices are in a small range, due to this they are well-compressed by entropyencodingand standard compression algorithms. htage3. -ompressionusingawell-known arithmeticencoding.

- I86-

Test hyperspectral data

To determine the effectiveness of the proposed algorithm in terms of compression degree, and also the limits of its applicability there have been conducted the experiments on hyperspectral AI (RSE Aviris). The AVIRIS system (Airborne Visible/Infrared Imaging Spectrometer) provides simultaneous adoption of the 224 spectral images with a wave length between 400 nm and 2500 nm. Also there has been done the comparison of the proposed algorithm with experimental results obtained for universal archivers (WinRar, WinZip and Lossles JPEG 2000 compressor) compression algorithms using an extension of JPEG compression standard, which is widely used in the commercial processing of RSE data. The experiments were done on a PC with IntelCore i5, 2.29 GHz and RAM of 4 GB with OS Windows 8.1 (Table).

Table. Characteristics of test hyperspectral AI

Channels number Image size Size(byte) Channels number Image size Size(byte)

100 50*50 1081600 200 50*50 2163200

100 100*100 4080400 200 100*100 8160800

100 200*200 16160400 200 200*200 32320800

100 300*300 36240400 200 300*300 72480800

100 400*400 64320400 200 400*400 128640800

100 614*512 125747200 200 614*512 251494400

150 50*50 1622400 224 50*50 2421632

150 100*100 6120600 224 100*100 9140096

150 200*200 24240600 224 200*200 36199296

150 300*300 54360600 224 300*300 81178496

150 400*400 96480600 224 400*400 144077696

150 614*512 188620800 224 614*512 281673728

The results of the experiments

The Fig. 1 shows the compression algorithms when varying the number of channels and the geometric size (50*50) in comparison with the universal archivers Winrar, 7Z, and JPEG Lossless. Indicators of compression degree D are 25% higher than universal algorithms through searching the main channel, which determines the priority of channels compression and taking into account interchannel correlation, indexed encoding.

To improve the efficiency of the algorithm, the experiments on the following parameters were done:

1) Cor - the given correlation value;

2) N - the number of AI channels group;

3) R - the size of AI channels;

4) K - the number of channels in the group N.

Fig. 2 shows that the indicators of the compression algorithm degree in R are higher in the compression degree than archivers WinRar, 7Z, and JPEG Lossless more than 80%.

4.5

4

c 3.5

J3 t

c

2.5

u

— !

=

0 1.5

2 1

0,5

0

- Correlation-

Algorithm I—Winrar

• JPEG

Lossless

Fig. 1. Lossless compression algorithm according to K

-~Jt~ 7 7

-Mr-^

4:5

4

0

1 3:5

5 3

VI 2.5

U

E. 2

C

o 1,5

0 3

1 1

0.5

0

50*50 LOO*10- 200*200 300*300 Fig. 2. Losslesscompressionalgori2hmsat various size

400*400

—•-CopLrLo:--

Eased Algorithm —Wirirar

-7Z

614*512

■■■■ JPEG Lossless

Geometric size

The ros^lohisoNhe expe^^me^ns baseo oir th^e compre ss^on algorithms dejjre e takinginto aecount the coreetatiod wnd rnfdrhnps show thattheeecdmmehdc dpaeometervdhtes arere foUows:

N = 2 - 10. This demonstrates that the subtraction (differential conversions) are effective with large range of channels in the group, then the average values of differences Sr are the least (Fig. 3), so this allows to store in the smallest amount on a disk space than the small amount of K in N.

In Fig4-5depictsthedependegoieson variousparameters of the experiments performed showing the modificatioTof rhecompres seonratio, forexamptri acN= s (D>4t,N = 9 (D^^X

According to chm easurtr df experiments duteeldn the ddpdndeade ed tllecocrelhli4n rrom the chosen channetsgroups 2nd 1he test xiNnhelit a revehiedthnlwhenthevn)ue Corer reducad then K is reducenptop4rteonhПy• Tnstnny theeffec)rvenott odShel^l^l^lios5t^d cempdntetonmeShodw ith the use of choosing the groups of the most correlated channels, there have been done the experiments on the number of filling channels, which show that when K = [5:50] the compression degree is higher than when K = [50:100], Fig. 6.

Conclusion

On the basis of the conducted researches the following conclusions should be drawn: the compression lossless algorithm considering intrabands correlation allows to increase the compression degree to (D>4,5) than in the universal arcdivers and JPEG Lossless algorithm; the proposed approach of finding the best groups of channels for a given correlation value has increased the efficiency of the

800

600

¿5 400

200

Fig. 3. Sr of thebest channelsand theis differences

0

— Ccirelation-Based Algorithm

— Wiiuai

0,7-0,75 0,75-0,8 0,8-0,85 0,85-0,87 0,88-1 Cor

GR1 GR2 GR3 GR4 GR5 N

50*50 50*50 50*50 50*50 50*50 R

15 IS 43 23 65 K

Fig. 4. Comparisonofloosiess comprs ssinntlooicthmsfoo N = f

use of channel subtraction phase (differential transformation); the obtained results of the comparison of the converted hyperspectral AI with archivers and JPEG 2000 Lossless allow to talk about the efficiency of the indexed encoding stages.

The perspectives for further researches can be encoding the transformed hyperspectral AI with an adaptive algorithm using regression analysis and a new model of interchannel correlation accounting.

♦ Correlation-Based Algorithm

—Я—Winrar

■ 7Z

Cor Ю"

h к

FT 5. Comparison oflosslesscompression forI^= 9

7 6,5 6 5,5 5

4,5 4

Q 3,5 3 2,5 2 1,5 1

0,5 0

/V

■50 каналов

100 каналов

■là— 150 каналов

• 220 каналов

5 7 9 11 13 15 17 19 22 25

5 7 10 12 13 14 15 17 19 20 21 24 25 27 29 31 33 35 37 39 40 42 45 50 55 60 65 70 72 75

3 5 7 10 13 15 18 20 22 25 27 30 32 33 35 37 39 40 42 45

3 5 7 9 10 12 14 16 18 20 22 24 25 27 28 30 35 37 39 40 42 45 47 49 50 55 58 62 67 70 75 77 80 83 85 87 89 90 92 95 98 100 1 05 1 10

Fig. 6. Tlo5 graplo moomberoMlHog tieeghpnirete

Referencen

[1] Yuan L., Jianping L., Ke G. Lossless compression of hyperspectral images using hybrid context prediction. Optiro Expnooo. 2012, 20(7), 8199-8206.

[2] Changguo L., Ke G. Lossless Compression of Hyperspectral Images Using Three -Stage PTd5cgoonw5fh PdrjitiveSeerch bhreshrlh.botofbhtipbhe Jpunroe of Sigroe Pnoroooibg, Imogo Pnoroooibg ordPorronb Rorogbiriob. 2014, 7(3), 305-316.

[3] Cheng-chen L., Yin-Tsung H. Lossless Compression ofHyperspectral Images Using Adaptive Prediction and Backward Search Schemes. Jpunroe of Iifonmorior Sriorro ord Ergiroonirg. 2011, 27, 419-435.

[4] AymanAeS;°wa E^Moliamed El. Sh. Hyperspectral Dad CompressionUslegW4ighted Spatial-Spectral Lossless Coding Technique. Ibrofboripboe Joumoe of Imogo Pnorooomg. 2012, 6(6), 468-477.

[5] Qiang Z., Pauca V.P., Plemmons R. Randomized methods in lossless compression of hyperspectrdX data. SPIE Joureal of Applied Remote Seesieg. 2013, 7, 1-16.

[6] Rnffaele P., Bruno —. On the —ompression of HyperspectrdX Data. IT CoNvergeece PRActice. 2013, 1(4), 24-38.

[7] —hangguo L., Ke G. Lossless —ompression of HyperspectrdX Images Using Three-Stage Prediction with Adaptive Search Threshold. Ietereatioeal Joureal of Sigeal Processieg, Image Processieg aed Pattere Recogeitioe. 2014, 7(3), 305-316. http://da.doi.org/ 10.14257/ ijsip. 2014.7.3.25

[8] Ganeshraj P., Sivasankar A. Scalable —ompression Method for Hyperspectral Images. Joureal of Eegieeerieg Scieeces. 2013, 2, 1-5.

[9] Diego V., Enrico A. Novel Rate Coetrol Algorithm for Oeboard Predictive Codieg of Multispectral aed Hyperspectral Images. 2014. Source: URL: http://araiv.org/pdf/1401.3277.pdf. DOI: 10.1109/ TGRS. 2013.2296329.

[10] Rnffaele P., Bruno —. Band —lustering for the Lossless —ompression of AVIRIS Hyperspectral Images. A—EEE Int. J. oe Sigeal aed Image Processieg, 2014, 5(1), 1-14.

[11] Algorithm for compressing hyperspectral aerospace images of earth remote sensing / Sarinova A.Zh., Zamyatin A.V. XI Ietereatioeal scieetific-practical coefereece of studeets, graduate studeets aedyoueg scieetists "youth aed modere ieformatioe teleologies'". 2013, 384-386.

[12] Zamyatin A.V., Sarinova A.Zh. Algorithm for compressing hyperspectrdX aerospace images with byte processing and interband correlation. AppliedIeformatics. 2013, 5(47), 37-42.

[13] Sarinova A. Lossless compression of hyperspectral images with pre-byte processing and infra-bands correlation. Sarinova A., Zamyatin A., —abral P. Dyea-Colombia. 2015, 82(190), 166-172.

[14] Sarinova A. Preparatory processing of algorithms for compressing hyperspectral aerospace images with allowance for band-to-band correlation with losses and without losses. Proceediegs of the XV Ietereatioeal Coefereece eamedafter A.F. Terpugova. Ieformatioe Techeologies aed Mathematical Modelieg. 2016, 123-128.

[15] Zamyatin A.V. Processing the algorithms, compression of hyperspectral images based on aerospace crossband correlation with and without lossles. Zamyatin A.V., Sarinova A.Zh. // Materials of the Nieeteeeth Ietereatioeal Scieetific Coefereece: ie 3 volumes. ueder the Society. Ed. VM VisheevsHy aed KE Samuilova Distributed computer aed telecommueicatioe eetworHs: coetrol, calculatioe, commueicatioe. (D——N-2016). 2016, 291-298.