SUPERPIXEL CLASSIFICATION METHOD BASED ON SEMI-SUPERVISED K-SVD AND MULTISCALE SPARSE REPRESENTATION

This application claims priority to Chinese application no. 201810757826.0 with a filing date of Jul. 11, 2018. The content of the aforementioned application, including any intervening amendments thereto, are incorporated herein by reference.

The present invention relates to the field of hyperspectral image classification technology, and in particular to a superpixel classification method based on semi-supervised K-SVD and multiscale sparse representation.

Over the years, remote sensing image classification has played an important role in many applications such as environmental damage assessment, crop growth regulation, land use monitoring, urban planning and reconnaissance. Compared with a single-band full-color image and a multispectral image, a Hyperspectral Image (HSI) can be used for detecting and distinguishing objects with higher accuracy because HSI has higher spectral resolution.

In a hyperspectral image, the spectral data of each pixel is a high-dimensional vector, and hundreds of data dimensions represent the spectral response of hundreds of bands. Hyperspectral image classification is mainly to classify each pixel based on spectral information. To achieve this goal, many pixel-level classifiers have been developed, including Support Vector Machine (SVM), support vector condition stochastic classifier, neural network, etc. Although these classifiers can make full use of the spectral information of HSI, they have not taken into account the spatial context, so noise often appears in the classification results. To solve this problem, and as a result that the pixels in a local area usually represent the same material and have similar spectral characteristics, many methods to obtain classification effect by integrating the information of near space have been developed. However, due to the lack of understanding of the near area, some risks exist in this kind of rough near area selection. Therefore, object-level classification appeared later. Although ground objects are presegmented in object-level classification, the classification effect is not ideal due to the existence of the problem of undersegmentation.

Inspired by the sparse coding mechanism of human visual system, Bruckstein first proposed the concept of sparse representation. In the field of hyperspectral image classification, the research of sparse representation is mainly focused on the acquisition of overcomplete dictionary and sparse solution:

In the field of sparse representation, the completeness of the sparse representation of the original signal is guaranteed by the overcomplete dictionary. The acquisition methods of the overcomplete dictionary are mainly divided into two types: methods based on mathematical models and methods based on training samples, wherein a dictionary acquisition method based on training samples inherently contains rich original signal characteristics, and direct splicing of the original signal is the most classical overcomplete dictionary based on training samples. However, due to the randomness of training samples and the rigidity of the dictionary, the completeness of the dictionary cannot be verified or improved. Therefore, dictionary learning methods have been proposed, among which, K-SVD dictionary learning method aims to minimize signal reconstruction errors, and dictionary update and sparse coding are carried out alternately through Orthogonal Matching Pursuit (OMP) and Singular Value Decomposition (SVD). Although the K-SVD method has strong universality and popularity, it has not emphasized the characteristics between different classes in the process of classification application.

For the researches on solving the sparse representation of the original signal, the most classic ones are Matching Pursuit (MP) and Orthogonal Matching Pursuit (OMP). In MP and OMP, the solution of sparse representation is based on a signal (pixel) and the influence of spatial context is not considered. Based on this, Joint Sparse Model (JSM) and Synchronous Orthogonal Matching Pursuit (SOMP) appeared subsequently. However, many problems in the selection of near space still exists in these two algorithms: on one hand, the shape of the near space is rectangular, the ground objects in the rectangular window are unknown, and the ground objects in the rectangular window are assumed to be unified in the algorithms, so this assumption becomes very dangerous when the scale is large; on the other hand, the scale of the near space area is single and needs to be set in advance, and different application environments have different optimal scales, so it is very difficult to configure this scale.

Therefore, how to provide a superpixel classification method based on semi-supervised K-SVD and multiscale sparse representation is an urgent problem to be solved by those skilled in the art.

In view of this, the present invention adopts training sample expansion based on superpixel and coefficient solution based on joint sparse model, introduces superpixel spatial information, and select a near space area that is more consistent with the real ground objects, so the noise in the same ground object is eliminated better and the classification effect is improved effectively.

To achieve the above purpose, the present invention adopts the following technical solution: a superpixel classification method based on semi-supervised K-SVD and multiscale sparse representation, in which software test results are displayed by the monitor, comprising the following steps:

step 1: carrying out semi-supervised K-SVD dictionary learning on the training samples of a hyperspectral image, and thus to obtain an overcomplete dictionary;

step 2: using the training samples and the overcomplete dictionary as the input to obtain the multiscale sparse solution of superpixels, and thus to obtain the sparse representation coefficient matrix of the training samples; step 3, using the obtained sparse representation coefficient matrix and overcomplete dictionary to obtain the result of superpixel classification by residual method and superpixel voting mechanism.

Preferably, the step 1 specifically comprises:

1) using the training samples of the hyperspectral image as the initial dictionary, and denoting the superpixel where the training samples x_pare located as X_p^ex=[x_p¹, . . . , x_p^Sp] ∈R^B×Sp, wherein S_prepresents number of pixels contained in the superpixel X_p^ex, x_p^s(1≤s≤S_p) represents an unlabeled sample in the superpixel X_p^ex; expanding each of the training samples x_pto obtain a new training sample set

X^ex[X₁^ex, . . . , X_P^ex]∈R^B×Pex

wherein P^ex=Σ_p=1^PS_prepresents the total number of the new samples;

2) each of the training samples x_phaving a corresponding label l_p(1≤l_p≤U), wherein U is the number of classes, and except that the training samples x_phave labels l_p, the labels of other samples in the superpixel X_p^exare regarded as l_p.

Preferably, a joint sparse model is adopted in the step 2 to obtain the sparse solution during sparse dictionary learning process, which specifically comprises:

1) assuming all samples with labels X^exin the new training sample set X^exas one set, wherein 1≤u≤U,

X_u=[x₁, . . . , x_c, . . . , x_cu] ∈R^B×C_u

X_uis a B×C_umatrix and represents a set composed of C_upixels, the set is a set of all samples with labels u in X^ex, and each pixel x_ccontains B spectral layers, wherein 1≤c≤C_u; the set satisfies that

{X_u}₁^P={X_u⊆X^ex|L(X_u)=u}

wherein L(X_u) represents a function for obtaining X_ulabels;

2) for a plurality of pixels in basically the same class of ground object materials, regarding the plurality of pixels as locally smooth and sparse areas;

3) sharing a common sparse mode by X_uof the same class of labels, and then C_ppixels can approximate the sparse representation through the atomic combination of the overcomplete dictionary:

X_u=[x₁, . . . , x_c, . . . x_Cu]≈D[a₁, . . . a_Cu]

wherein D ∈ R^B×ND is the overcomplete dictionary with N_Datoms and labels corresponding to each atom, and the atoms are the column vectors in the overcomplete dictionary D;

A_u∈ R^ND×CP is the matrix of sparse coefficient [a₁, . . . , a_Cu]; due to local smoothness and sparsity, all sparse coefficient vectors {a_c}₁^Cu are assumed to have the same non-zero row position if A_uhas only L_C(L_C<<N_D) non-zero rows therein;

4) solving A_uby solving the joint sparse recovery problem:

=arg min_A∥X_u−DA_u∥₂²,

∥A_u∥_row,0≤L_C

wherein ∥A_u∥_row,0is l₀norm obtained by rows, and is the number of non-zero rows of A_u.

Preferably, the joint sparse recovery problem is solved by orthogonal matching pursuit, and at this point, the overcomplete dictionary D is normalized by columns to make the l₂norm unitized.

Preferably, the step 3 specifically comprises:

1) giving a test pixel y₁in a hyperspectral image, selecting T adjacent areas of different scales/dimensions, and arranging the pixels in the selected area to construct the corresponding multiscale matrix Y^multiscale=[Y₁, . . . , Y_t, . . . , Y_T], wherein Y_tincludes the pixels from the t^thscale area;

2) rewritting the sparse representation problem of T scales as

{A^}t=1T=argmin{At}∑t=1TYt-DAtFAtrow,0≤K∀1≤t≤T

wherein [A₁, . . . , A_t, . . . , A_T] is the sparse coefficient of [Y₁, . . . , Y_t, . . . , Y_T], which constitutes the multiscale sparse representation matrix A^muitiscale;

3) creating an adaptive set L_h, wherein h=1, 2, . . . is expressed as the index of a set of non-zero scalar coefficients and belongs to the same class in the multiscale sparse representation matrix A^multiscale; for columns in each scale of A^multiscale, indexes of adaptive set L_hare in the same row;

4) determining the class of the test pixel y₁based on the minimum total representing error

c^=argmincYmultiscale-DcA^cmultiscaleF

Â_c^multiscalerepresents the row corresponding to class c in Â^multiscale.

Preferably, the sparse representation problem of T scales is solved by orthogonal matching pursuit.

Preferably, the steps to select the adaptive set in step 3) are as follows:

(1) finding a representing atom with the minimum residual error for each scale of each class;

(2) combining the optimum atoms of all scales of each class into a cluster;

(3) selecting the optimum cluster from clusters of all the classes and recording the index of atoms in the cluster as the adaptive set.

Preferably, to combine and classify the information of different scales, the joint sparse constraint l_{row, 0}is applied to solve the multiscale sparse representation matrix A^multiscale, which enables the sparse coefficients of different scales share the same sparse mode, i.e., the same atom set is selected to represent pixels of different scales; at this point, A^multiscaleis jointly restored by solving the following problem:

Â^multiscale=arg min_Amultiscale∥Y^multiscale−DA^multiscale∥_F

∥A^multiscale∥_{row, 0}≤K

Preferably, the sparse coefficient required by the multiscale pixel need to be beneficial to the sparse mode of the same class level, but at the same time sparse modes of different scale levels are allowed within each class.

Preferably, the method further comprises the following step after determining the class of the test pixel y₁in step 4): performing the superpixel voting process once again by using the superpixels with the least scale.

It is known from the above technical solution that the present invention provides a superpixel classification method based on semi-supervised K-SVD and multiscale sparse representation, having the following beneficial effects:

The present invention has good capacity to remove salt and pepper noise and enrich training samples;

The present invention can provide very stable classification results in the condition with various sample sizes, and with the enrichment of samples, the classification method provided by the invention has higher and higher classification performance;

The proposing of the method is of great significance to solving the problem of salt and pepper noise and the problem of high dimension and small samples in the field of hyperspectral image classification, as well as the problem of how to effectively use space information in classification algorithm based on sparse representation.

The technical solution in the embodiments of the present invention will be clearly and fully described below in combination with the drawings in the embodiments of the present invention. Apparently, the described embodiments are merely part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those ordinary skilled in the art without contributing creative labor will belong to the protection scope of the present invention.

The algorithm of the present invention comprises two parts: semi-supervised K-SVD dictionary learning algorithm and multiscale sparse representation solution based on superpixels. Firstly, the training samples of a hyperspectral image are given for semi-supervised K-SVD dictionary learning, and thus to obtain an overcomplete dictionary with distinct characteristics; secondly, test samples and the overcomplete dictionary are taken as the input to introduce the algorithm of multiscale sparse representation solution of superpixel spatial information, and thus to obtain the sparse representation coefficient matrix of the test samples; finally, the obtained sparse representation coefficient matrix and overcomplete dictionary are used to obtain the final result of superpixel classification by residual method and superpixel voting mechanism.

In K-SVD algorithm, we need a dictionary D composed of unitized atoms and the sparse representation coefficient of a signal relative to the current dictionary, and we hope to minimize the reconstruction errors in each update:

{{circumflex over (D)}, Â^train}=argmin∥X^train−DA^train∥₂², s.t. ∥a_p^train∥₀≤L_D  (1)

p=1, . . . , P,

wherein D=[d₁, . . . , d_ND] ∈ R^B×ND is the dictionary with N_Datoms to be learned; X^train=[x^train₁, . . . , x_P^train] ∈R^B×Pis a training sample set containing P training samples; A^train=[a₁^train, . . . , a_P^train] ∈R^ND×Pis the sparse representation matrix corresponding to X^train, and L_D(L_D<<N_D) is the upper boundary of model sparsity. K-SVD is composed of a sparse coding stage and a dictionary update stage: firstly, the sparse representation matrix A^trainin formula (1) is solved by fixed dictionary D and OMP algorithm.

The second stage is started once A^trainis obtained, all the other atoms in the dictionary are fixed, the current atom is updated by SVD decomposition of error terms, and the second stage is ended when all the atoms in the dictionary are updated. Assume that the k^thcolumn of atoms are currently updated, denote the k^thcolumn as d_k, and denote the corresponding k^throw in the sparse matrix A^trainas a_T^k, so the sample matrix and dictionary approximation error after d_ka_T^kis removed is

Xtrain-DAtrainF2=Xtrain-∑j=1NDdjaTjF2=(Xtrain-∑j≠kNDdjaTj)-dkaTkF2=Ek-dkaTkF2(2)

After the current error E_kis obtained, we only need to adjust d_kand a_T^kto make the error between the product thereof and E_ksmall enough. To ensure sparsity, we only extract non-zero terms from a_T^kto constitute the column vector a_R^k, record the indexes of the columns of a_T^knon-zero terms, extract corresponding column vectors from E_kaccording to the indexes to constitute matrix E_k^R, perform SVD decomposition of E_k^Rto obtain E_k^R=UΔV^T, update d_kby using the first column of U, and update a_R^kby using the result of Δ(1,1)×V.

These two stages are iterated until certain stopping criteria are met.

Two problems are found in applying the traditional K-SVD algorithm to hyperspectral image classification: on one hand, in K-SVD algorithm, training samples are limited to the total number of original samples, sufficient information is not available for small sample classes, but the number of labeled samples is limited, so how to effectively increase the number of training samples is worth studying; on the other hand, the concept of class does not exist in K-SVD algorithm, and how to enlarge the properties of each class by sparse representation is a major research topic in the field of sparse representation.

To solve these two problems, we expand the training samples of K-SVD algorithm based on superpixels, and expand the K-SVD algorithm without supervised learning to the semi-supervised K-SVD algorithm; and optimize OMP solution method during dictionary update to a solution method based on Joint Sparse Model (JSM).

For the K-SVD method, both the training samples and the initial dictionary are original training samples. However, for a hyperspectral image, the numbers of labeled samples in different classes are different or even vary greatly from the each other; therefore, small sample classes usually have the problem of insufficient samples. To avoid this problem without increasing labeled training samples, we use a semi-supervised strategy to expand the training samples based on superpixels.

The scale of superpixel segmentation is different from that of traditional object-level segmentation; compared with the target of capturing a complete ground object, the superpixel focuses more on one part of the ground object accurately, and then covers the whole ground object completely through the combination of several superpixels. Such excessive segmentation scale enables a single segmentation area (superpixel) to more accurately match the boundary of the ground object locally. Therefore, for an ideal superpixel, we can assume that all of the pixels therein belong to the same ground object.

If the original training sample is X^train=[x₁^train, . . . , x_P^train] ∈ R^B×P, P training samples (pixels) x_p(1≤p≤P) are included, and B spectral layers contained in each pixel x_pare expressed by the B dimensional vector. Each of the training samples has a corresponding label l_p(1≤l_p≤U), wherein U is the number of classes; therefore, the label set is L^train=[l₁^train, . . . , l_P^train] ∈ R^p. The training samples and labels thereof are the basis for final classification, in other words, how to obtain a better classification result on the basis of limited number of labeled training samples is the objective of the study.

To prevent the atomic number of the dictionary from being out of control, the original training samples are used as the initial dictionary. However, new training samples will be added as the final training samples. The superpixel where the training samples x_pare located is denoted as X_p^ex=[x_p¹, . . . , x_p^Sp] ∈ R^B×Sp, wherein S_prepresents number of pixels contained in the superpixel X_p^ex, and x_p^s(1≤s≤S_p) represents an unlabeled sample (pixel) in the superpixel. By expanding each training sample x_p, a new training sample set is obtained:

X^ex=[X₁^ex, . . . , X_P^ex] ∈ R^B×Pex   (3)

wherein P^ex=Σ_p=1^PS_pis the total number of the new samples. As mentioned before, except that original training samples x_phave labels l_p, the labels of other samples in the superpixel can be regarded as l_p. Therefore, without increasing the number of labeled training samples, the number of training samples can be greatly increased by the semi-supervised policy of introducing a great number of unlabeled samples whose classes can be deduced.

In the K-SVD algorithm, we need to obtain a new sparse representation according to the new dictionary after each update of the dictionary completes, but the OMP algorithm used by K-SVD does not have a concept of classification, and the sparse solution processes of samples of the same class are isolated from one another. However, in the field of classification of sparse representations, ideal sparse representations of samples of the same class shall be highly similar, and therefore, the Joint Sparse Model (JSM) is introduced for the sparse solution during the dictionary learning process. The solution process is as follows.

Taking class u(1≤u≤U) as an example, we arrange all the samples with labels u in X^exinto one set

X_u=[x₁, . . . , x_c, . . . , x_Cu] ∈ R^B×Cu   (4)

X_uis a B×C_umatrix and represents a set composed of C_upixels, the set is a set of all samples with labels u in X^ex, and each pixel x_c(1≤c≤C_u) contains B spectral layers. Therefore, the set satisfies

{X_u}₁^P={X_u⊆X^ex|L(X_u)=u}  (5)

wherein L(X_u) represents a function for obtaining X_ulabels. In an HSI, when a plurality of pixels have basically the same class of ground object materials, the plurality of pixels can be regarded as locally smooth and sparse areas. With the JSM concept, we make X_uwith the same class of labels share a common sparse mode. C_ppixels can approximate the sparse representation through the atomic combination of the given dictionary:

X_u=[x₁, . . . , x_c, . . . , x_Cu]≈D[a₁, . . . , a_Cu]  (6)

wherein D ∈ R^B×ND is a dictionary with N_Datoms and labels corresponding to each atom, the atoms are column vectors in the overcomplete dictionary D, for example, a₁=[1,0,0,0,0 . . . ,1] in formula (6), x₁is the combination of the first atom and the last atom in the overcomplete dictionary D.

A_u∈ R^ND×Cp is the matrix of sparse coefficient [a₁, . . . , a_Cu]. Due to local smoothness and sparsity, it can be assumed that A_uhas only L_C(L_C<<N_D) non-zero rows. The Joint Sparse Model (JSM) indicates that all sparse coefficient vectors {a_c}₁^Cu have the same non-zero row position.

The JSM solution A_uis achieved by solving the joint sparse recovery problem:

=argminAXu-DAu22,Aurow,0≤LC(7)

wherein ∥A_u∥_row,0is the l₀norm obtained by rows, and is the number of non-zero rows of A_u. This problem can be solved by Synchronous Orthogonal Matching Pursuit (SOMP). When the problem is solved by SOMP, the dictionary D is normalized by columns to make the l₂norm unitized.

In the traditional K-SVD algorithm, the label of the training sample is not used, and the update of the dictionary only focuses on how to obtain a smaller error in each column. The semi-supervised K-SVD algorithm herein uses the semi-supervised learning policy: first, obtaining expanded training samples by using the same assumption of the same superpixel label on the basis of the fundamental training sample; and then implementing the means of obtaining a joint sparse representation matrix for each class of expanded training samples to use the labels of training samples and a great number of unlabeled samples during the learning of the dictionary.

Herein, we present a complete learning process of the semi-supervised K-SVD dictionary in the form of pseudocodes.

Unlike the sparse solution of the training sample in the learning process of dictionary, the sparse solution of the test sample shall be performed in a more cautious and stable way, and a more purposive design is needed. The practical SOMP algorithm described above takes the pixels in the near space (preset 3×3, 5×5 or 7×7 area) of the target pixel into account in the process of sparse solution, and the prior art solves the problem of fixed and single SOMP area and proposes a multiscale SOMP method. The method jointly considers pixels in multiscale near space of the target pixel (e.g., joint use of near space of three scales: 3×3, 5×5 and 7×7) to obtain more stable and abundant space context information. However, the problem of hidebound selection of near space remains unsolved, and therefore, we design the superpixel classification algorithm of multiscale sparse representation that is closer to actual ground object, has smaller noise and uses the superpixel space information.

First, we usually can obtain superpixel split images of three scales that each superpixel contains an average of about 16, 64 and 256 pixels by using the existing superpixel split algorithm. In these images, images with small pixels (a small number of pixels) provides a more accurate local coverage for the ground object, and images with large pixels (a large number of pixels) can contain more abundant space context. It is found by experiment that the scale span of superpixel is larger than the rectangular window. The single use of superpixel as the spatial information leads to a less-than-ideal result due to too few auxiliary areas, and single use of rectangular window can result in unknown risk of ground object. Therefore, we decide to cross use the two space scales to obtain the most accurate and abundant space information.

By giving a test pixel y₁in HSI and selecting T adjacent areas of different scales (dimensions), pixels in the selected area can be arranged to construct the corresponding multiscale matrix Y^multiscale=[Y₁, . . . , Y_t, . . . , Y_T], wherein Y_tincludes the pixels from scale area t. In hyperspectral images, areas of different sizes often exhibit different spatial structures and features. However, because all different scales correspond to the same test pixel y₁, supplementary and pertinent information shall be provided to classify the y₁more accurately.

It is assumed that we have a structure dictionary D and a multiscale matrix [Y₁, . . . , Y_t, . . . , Y_T] for the test pixel y₁. Then the sparse representation problem of T scales can be rewritten as

{A^}t=1T=argmin{At}∑t=1TYt-DAtFAtrow,0≤K∀1≤t≤T(8)

wherein [A₁, . . . , A_t, . . . , A_T] is the sparse coefficient of [Y₁, . . . , Y_t, . . . , Y_T], which can constitute the multiscale sparse representation matrix A^multiscale. FIG. 1 of the Description shows the principle diagrams of three kinds of sparse models, and this problem can be solved by applying the SOMP algorithm at every scale. See FIG. 1(a). However, such separate policy does not consider the strong relevance among different scales. To combine and classify the information of different scales, the joint sparse constraint l_{row, 0}can also be applied to the multiscale sparse representation matrix A^multiscale, which guarantees that the sparse coefficients of different scales share the same sparse mode, i.e., the same atom set is selected to represent pixels of different scales. See FIG. 1(b). Under this assumption, A^multiscalecan be jointly restored by solving the following problem:

A^multiscale=argminAmultiscaleYmultiscale-DAmultiscaleFAmultiscalerow,0≤K(9)

The above-mentioned joint method can mix information coming from different scales for classification. However, it is assumed that pixels of all scales are of the same sparse mode, but because structure features of different scales of HSI differ from one another, the restriction is too strong. In other words, to obtain a better representation, pixels from each scale must have “freedom” to select different suitable atoms. Furthermore, because all the multiscale information corresponds to the same test pixel y₁, the “freedom” of selecting the atoms shall be confined to one level. Therefore, the sparse coefficient required by the multiscale pixel shall be beneficial to the sparse mode of the same class level, but at the same time sparse modes of different scale levels are allowed within each class.

According to the above thought, a flexible model is adopted herein. The model achieves the flexible selection process of atoms by introducing an adaptive sparse policy. An important part of this policy is the adoption of adaptive set. For each adaptive set L_h, h=1, 2, . . . is expressed as the index of a set of non-zero scalar coefficients, belonging to the same class in the multiscale sparse matrix A^multiscale. For columns in each scale of A^multiscaleindexes of adaptive set L_hare in the same row. This is achieved by applying joint sparse regularization to pixels within each proportion. In addition, L_hat different scales can be in different rows. This means that pixels at different scales can freely select different atoms of each class. In this way, sparse coefficients of different scales can vary but still belong to the same class as shown in FIG. 1(c).

By combining the adaptive set and the l_{row, 0}norm, a new adaptive norm l_{adaptive, 0}is created on the A^multiscale, which can select a few adaptive sets from A^multiscale. Then, A^multiscalematrix can be obtained through the adaptive norm l_{adaptive, 0}, i.e.,

A^multiscale=argminAmultiscaleYmultiscale-DAmultiscaleFAmultiscaleadaptive,0≤K(10)

To solve the problem with formula (10), the multiscale SOMP algorithm is used.

The biggest difference between the multiscale SOMP algorithm and the OMP and SOMP is how to select a new adaptive set. In the multiscale SOMP algorithm, it is hoped that the selected adaptive set shall meet the following requirements: 1. the ordinal number of rows in which each column of non-zero terms of adaptive subsets of the same scale exists shall be the same; 2. the number of non-zero rows of adaptive subsets of different scales in the same class shall be the same. See FIG. 1(c).

To achieve this, according to FIG. 1(c), the steps in selecting a new adaptive set are as follows: 1) finding a representing atom with the minimum residual error for each scale of each class; 2) combining the optimum atoms of all scales of each class into a cluster; 3) selecting the optimum cluster from clusters of all the classes and recording the index of atoms in the cluster as the new adaptive set.

After the multiscale sparse representation matrix A^multiscaleis obtained, the class of the test pixel y₁can be determined based on the minimum total representing error.

c^=argmincYmultiscale-DcA^cmultiscaleF(11)

Â_c^multiscalerepresents the row corresponding to class c in Â^multiscale.

So far at this moment, the classification result of HSI sparse representation in pixels can be obtained. However, because the data dimension of spectral information is too high, the classification at the pixel level usually has certain salt and pepper noise interference. To further smooth the classification result and improve the classification accuracy, this algorithm will perform the superpixel voting process once again by using the superpixels with the least scale.

Taking superpixel S₁as an example, because the randomness of allocation of training samples and test samples, each superpixel may contain these two kinds of samples, the superpixel S₁containing M training samples and N test samples can be expressed as

S1=[s1,…,sM,sM+1,…,sM+N]=[xs1,…,xsM,ysM+1,…,ysM+N]∈RB×(M+N)(12)

wherein s_i(1≤i≤M+N) is a certain pixel of the superpixel; when s_iis the training sample, s_i=x_si (1≤i≤M); and when s_iis the test sample, s_i=y_si (M+1≤i≤M+N). The label set of the superpixel is

L₁=[l₁, . . . , l_M, l_M+1, . . . , l_N] ∈ R^N  (13)

Similarly, l_i(1≤i≤M+N) is the label of pixel s_i; when s_iis the training sample, l_iis the real ground object label of x_si; and when s_iis the test sample, l_iis the original classification result of y_si.

Based on the assumption that all pixels within the superpixel belong to the same ground object, the classification result of all the test samples is obtained once again by voting, and the method is as follows:

{l_i}_M+1^M+N=l_maxnum=maxnum(L₁)   (14)

wherein the function maxnum(L₁) is used to obtain the label value l_maxnumthat occurs most often in the L₁vector. The final superpixel classification algorithm of multiscale sparse representation is obtained.

We provide the process of complete superpixel classification including the dictionary learning in the form of pseudocodes.

To verify the effectiveness and generality of the algorithm, the performance contrast experiment is performed on three data sets including Salinas, Indian Pines and Pavia University. Through objective measures of OA, AA and Kappa coefficient, the target algorithm is compared with various classic or advanced algorithms mentioned herein to obtain the objective evaluation of the proposed algorithm.

The hardware platform for the experiment is: Intel core i5-6300HQ CPU, and 4 GB memory. The target algorithm, the comparison algorithm, and the calculation of OA, AA and Kappa coefficient are all realized with codes by using the matlab 2014a Community Edition.

The Salinas data set is collected by the AVIRIS sensor from the Salinas Valley of California. The image dimension is 512*217*224, and the spatial resolution is 3.7 meters per pixel. Similar to the Indian pines data set, 20 spectral bands absorbed by the water bodies are removed (S/N: 108-112, 154-167 and 224), and the reference image has 16 different classes. The real information of the ground object is shown in FIG. 2.

The Indian pines data set is obtained by Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor system over the Indian pines test station at northwest Indiana of the United States. The data size is 145×145 pixels, the image contains 200 pieces of wave band spectral information after the noise wave band is removed, and the resolution is 20 m. The real ground object has 16 marked areas, and the real information of the ground object is shown in FIG. 3.

ROSIS-03 sensor records the Pavia University data set which captures the urban area surrounding the Pavia University of Italy. The image dimension is 610*340*115, the spatial resolution is 1.3 meters per pixel, and the spectral coverage range is 0.43 to 0.86 micrometer. The data set abandons 12 very noisy passages, and marks 9 information classes for this image. The real information of the ground object is shown in FIG. 4.

Three objective indexes (i.e., overall accuracy (OA), average accuracy (AA) and Kappa coefficient) are used during the experiment to evaluate the quality of classification results. OA measures the percentage of correctly classified pixels. AA represents the mean value of percentages of correctly classified pixels of various classes. Kappa coefficient estimates the proportion of correctly classified pixels via the confusion matrix.

In the contrast experiment of the three data sets, a scale allocation standard needs to be set for the target algorithm. Eventually, 8 layers of scale space are set for Y^multiscaleincluding, from small to large, 3×3 rectangular frames, superpixels with an average of 16 pixels, 7×7 rectangular frames, superpixels with an average of 64 pixels, 11×11 rectangular frames, 13×13 rectangular frames, 15×15 rectangular frames, and superpixels with an average of 256 pixels. The sparseness S=10, and degree of learning K=5. The reference contrast algorithm is set according to the recommended algorithm parameter.

To fully verify the effectiveness of the algorithm, the contrast algorithms selected include JSM, non-local weighted sparse representation (NLW-SR), single-scale adaptive sparse representation (SASR), multiscale separation sparse representation (MSSR), multiscale rigid joint sparse representation (MJSR) and multiscale flexible joint sparse representation (MASK),wherein the SASR, MSSR and MJSR algorithms are sparse solving algorithms of multiple joint sparse models mentioned in the MASK algorithm literature.

To verify the performance of algorithm on different sample quantities, the Salinas, Indian Pines and Pavia University data sets are respectively used to imitate the situations with normal sample quantity, less sample quantity and adequate sample quantity.

In the Salinas data set, due to the adequate sample quantity, only 1% of all the marked pixels are randomly selected as the training samples, and remaining 99% pixels are used as test samples, and the actual allocation of sample quantities is shown in Table 1.

It can be seen from Table 1 that in the Salinas data set, the quantity of samples of different classes is small, and the samples are evenly distributed. To verify the advantages of the SK-MSR algorithm provided herein on the small set of samples, fewer training samples are selected for the test.

It can be seen from Table 2 that in the Salinas data set, the SK-MSR algorithm generally has the most excellent classification accuracy, and shows the most stable state regardless of the relative size of sample quantity, thus the average classification accuracy is the highest. The excellent performance on Kappa coefficient that quantitatively measures the classification accuracy via the confusion matrix more adequately shows the stability and accuracy of Salinas in terms of large data amount. The classification results of Table 2 are the average data of operating 10 by various algorithms.

It can be seen from FIG. 5 that the SK-MSR algorithm basically realize the correct classification of Salinas data set, the scale of superpixel guarantees the removal of salt and pepper noise, and the border is relatively intact.

According to the characteristics of the Indian pines data set, that data set is used to verify the performance of the comparison algorithm and the target algorithm when the training samples are not adequate. 10% of all the marked pixels are randomly selected as the training samples, remaining 90% pixels are used as test samples, and the actual allocation of sample quantities is shown in Table 3.

It can be seen from Table 3 that in the Indian Pines data set, the sample quantities of different classes vary widely, and typical high dimension and small sample signals exist. In many algorithms, the problem of small sample arises.

It can be seen from Table 4 that due to certain sample expansion, the SK-MSR algorithm of the present invention has good performance in small sample classes 1, 7 and 9. However, due to high degree of similarity among the Indian Pines data set samples (basically the plant samples) and mosaic distribution of different classes (as shown in FIG. 3), the overall performance of SK-MSR is not as good as that of the MASR algorithm, but is better than those of other contrast algorithms. It is observed that the SK-MSR algorithm is less suitable for the class that lacks in samples, but has relatively good and stable performance on the data set of that class and is relatively good at processing high dimension and small sample signals. The classification results of Table 4 are the average data of operating 10 by various algorithms.

It can be seen from FIG. 6 that due to the mosaic distribution of similar samples in the Indian Pines data set, the SK-MSR algorithm basically guarantees the correct classification in case of insufficient training samples, but the border regions of similar samples cannot be split out with utmost accuracy and precision.

To prove the performance of the SK-MSR algorithm under circumstances of adequate training samples and large data amount of test samples, the Pavia University data set is selected once again for verification. 300 marked pixels of each class are randomly selected as the training samples, the remaining pixels are used as the test samples, and the actual allocation of sample quantities is shown in Table 5.

Under this kind of training sample selection, all the algorithms shall be able to generate a sufficiently complete dictionary, and sufficient to fully verify the performance of the contrast algorithm and the SK-MSR target algorithm.

It can be seen from Table 6 that when abundant training samples can be extracted from the data set, the SK-MSR algorithm has obviously better classification accuracy than other algorithms. The classification results of Table 6 are the average data of operating 10 by various algorithms.

According to FIG. 7, it is clear that the SK-MSR algorithm basically achieves exactly correct segmentation on Pavia University, and abundant training samples ensures that the algorithm has excellent correct classification result.

By performance comparison on the three data sets, it is observed that, from a comprehensive perspective, the SK-MSR algorithm has the most stable performance and the highest classification accuracy among all the algorithms used during the experiment. In the situations with normal sample quantity, less sample quantity and adequate sample quantity, the SK-MSR algorithm has stable and high classification accuracy, and particularly with the increase in the sample quantity, the advantages of the SK-MSR algorithm become increasingly apparent.

This embodiment provides a comprehensive introduction of the implementation process of semi-supervised K-SVD dictionary learning algorithm, and designs the superpixel classification algorithm of multiscale sparse representation that uses the superpixel spatial information according to the characteristics of hyperspectral images. After explaining the reason and the method of introducing superpixel spatial information, this embodiment describes the complete algorithm implementation process. Finally, the contrast experiment on multiple data sets confirms the technical effect of the present invention, that is the proposing of the algorithm is of great significance to solving the problem of salt and pepper noise and the problem of high dimension and small samples in the field of hyperspectral image classification, as well as the problem of how to effectively use space information in classification algorithm based on sparse representation.

Each embodiment in the description is described in a progressive way. The difference of each embodiment from each other is the focus of explanation. The same and similar parts among all of the embodiments can be referred to each other. For a device disclosed by the embodiments, because the device corresponds to a method disclosed by the embodiments, the device is simply described. Refer to the description of the method part for the related part.

The above description of the disclosed embodiments enables those skilled in the art to realize or use the present invention. Many modifications to these embodiments will be apparent to those skilled in the art. The general principle defined herein can be realized in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principle and novel features disclosed herein.