CC BY
28
MODELING OF INFORMATION SYSTEMS OF COMPUTER VISION FOR MACHINE TRANSLATION BY METHODS OF THE THEORY OF GRAPHS
DOI 10.24411/2072-8735-2018-10158
Andrey V. Alyoshintsev,
Moscow Technical University of communication and informatics, Moscow, Russia, alyoshintsev@mail.ru
Alexander N. Sak,
Moscow State University of Civil Engineering, Moscow, Russia, sak_inter@mail.ru
Keywords: computer vision, graph theory, subordination tree, component tree, binary graph, syntax, semantics, artificial intelligence.
Computer vision is a scientific direction in the artificial intelligence area (robotics involved in obtaining images of real-world objects, their processing and use of data to solve applied problems (without human participation)). The data representation (for example, original sentence or the connected image components) is the basis for both computer vision information systems and machine translation systems. Data can be represented by means of graphs, where words, groups of pixels or variants of decisions are used as vertices, and as edges is the relationship between words in a sentence, elements of images or decisions. In most sentences, the relations of subordination and linear order are related. The representation of the syntactic structure of the sentence in the form of a subordination tree is used in generative grammars of the language and in the algorithms of syntactic analysis. The tree is built, starting from the distribution of lexical units of the sentence by parts of speech, and then there is a transfer from the subordination tree to the tree of the components. A binary search tree is a kind of data structure that corresponds to the representation of the sentence in the form of a tree of components.
Information about authors:
Andrey V. Alyoshintsev, Moscow Technical University of communication and informatics, Moscow, Russia Alexander N. Sak, Moscow State University of Civil Engineering, Moscow, Russia
Для цитирования:
Алешинцев А.В., Сак А.Н. Моделирование информационных систем компьютерного зрения для машинного перевода методами теории графов // T-Comm: Телекоммуникации и транспорт. 2018. Том 12. №10. С. 58-63.
For citation:
Alyoshintsev A.V., Sak A.N. (2018). Modeling of information systems of computer vision for machine translation by methods of the theory of graphs. T-Comm, vol. 12, no.10, рр. 58-63.
W
Graphs are one of the most important areas of the theory of computing systems. This is an abstract concept through which you can describe a variety of real phenomena such as the organization of transport systems, human relationships, and representation of the data structure. In linguistics, graph theory solves many problems associated with the representation of formal relations between the components of a sentence.
Work at visual images and analysis of natural language sentences has many common approaches. For example, the classic algorithm for labeling connected components is based on a data structure appropriate to unify and search components. This data structure provides an effective implementation of the algorithm for labeling connected components. When analyzing images by means of graphs, pixels are grouped into connected components by using pixels with similar characteristics. Graphs are binary (such a structure provides the best way to store and process information). Nodes when building a graph for image processing are related as parent and child ones, which is close to the sentence structure, also expressed by a binary graph. Apart from grouping pixels, binary graphs are also used at a higher level corresponding to a decision-making process when analyzing graphic objects.
A binary decision tree is a data structure in the form of a binary tree, each node of which is bound to the decision choice function. The decision choice function is applied to an unknown feature vector and determines which child node of the current node should be processed further - the left one or the right one. A similar algorithm is observed when constructing a tree of components when analyzing sentences.
A correct description of the sentence by means of the graph allows to use to a certain extent, certain semantic links between lexemes apart from the syntactic ones,
When translating from English into Russian at the first stage of the sentence analysis, special attention is paid to the problem of polysemy / homonymy of lexical units in the sentence. In English, a set of words can have one meaning, but belong to different parts of speech like "to go" which is a verb and 'Lto have a go" is a noun, or be expressed with one part of speech, but can have many unrelated meanings like, for example, the verb 1Lto draw. For this reason, the problem of determining to which part of the speech each lexeme belongs in the sentence [l| is solved.
Each lexeme gets a set of parts of speech that the given word can match in the sentence. Every possible sentence, i.e. a combination of parts of speech, can be regarded as a path in a graph whose vertices are a complete set of possible parts of speech corresponding to a given word [2]. Each possible part of speech corresponding to the i-th word is connected by an edge with every possible part of speech corresponding to (i + I) -th word. Using the statistical data of linguistic environment, for example, a noun is less likely to join other nouns, if they do not perform the function of an adjective, which in turn must be reflected in the dictionary, you can assign a certain weight to each part of speech corresponding to the word. The cheapest path through this graph will be the best interpretation of the sentence in terms of determining its words belonging to parts of speech.
The determining of parts of speech of the sentence is the first step to interpret it and to find the subordination tree, i.e. hierarchical structure of the sentence.
The new play is a great draw
article-^adjective. verb —^srtricle—^adjective-iflouti
■¿floun--"" . verb
Fig. 1. Determining parts of speech by using a weighted graph
Several linguistic schools developed several completely different ways of describing the syntactic structure of a sentence. We are interested in two representations of the syntactic structure that can be expressed by means of graphs: a) a description of the sentence structure through the system of its word combinations, i.e. through the relations of subordination and b) a description of the sentence structure through a hierarchy of its immediate components.
The description of the sentence structure by using the subordination tree means dealing with the word-combination. By word-combination we mean a syntactically related group of sentence words consisting of two words, and its components can be separated in the sentence by other words. In the word-combination we distinguish the main and a subordinate word, thus setting the subordination tree. The subordination tree can be considered as an adequate representation of the syntactic structure of the sentence, if the sentence doesn't have any formal or semantic links not reflected in the tree. Such a representation is not always met and is called an ideal tree 13]. If the subordination tree is not considered to be ideally hierarchical, connections and dependencies not reflected in the tree must be specified separately. The representation of the syntactic structure of a sentence in the form of a subordination tree is used in building up generating grammars and in algorithms for syntactic analysis. It is convenient that the tree graph reflects not only the relation of subordination, but also the order relation corresponding to the linear arrangement of words in the sentence, which are connected in a certain way. This connection is called projectivity. Properties of the subordination tree: no two branches intersect each other and no branch derived from some point a intersects perpendiculars dropped from the top points.
As an example of a subordination tree, let's take the previous example with already determined parts of speech.
article adjective noun verb article adjective noun
Fig. 2, A subordination tree
The syntactic position of the identified parts of speech will allow us to build up a subordination tree, depending on, for example, the position of the verb with respect to names or nominal groups, as in our case, where one nominal group consisting of an article, an adjective and a noun stands before a verb, and the other comprised of similar components is placed after a verb. In accordance with the linear structure of the sentence, the first group should be considered as a subject, and the second one as the predicate of the sentence.
Representation of a sentence with a binary tree is good because in modern programming languages data is represented by linked lists that easily turn into binary trees and vice versa. Another advantage of the binary tree is a proven algorithm for comparing binary trees (a powerful too) for comparing the syntaetie structures of two sentences).
Let's compare the sentence with an arithmetic expression. An arithmetic expression is represented by operators and numbers corresponding to the terminal elements. The problem is to represent the arithmetic expression in this way by leading it to such a form, so that it can be compared with another arithmetic expression based on a formal surface structure (without recourse to calculations).
At the first stage, we use the algorithm to express an expression using a binary tree. First, we give each operator a certain weight;
for (int m=0;m<l 1 ;m++)
i
mas = mmm[i 1, m]; if(mas = "*")
r!inq.AddLast(new poini(mas, 4, s)); else
if (mas = "/")
rlinq.AddLast(new point(mas, 3, s}); else
if (mas ="+")
rlinq.AddLast(new point(mas, 5, s)); else
if (mas == "-")
rlinq.AddLast(ncw (mini(mas, 5, s)); else if(mas!="")
rlinq.AddLast(new point(mas, 1, s));
s++;
}
We define the operator with the greatest weight and its place in the expression. We split the expression into the left and right sides with respect to the operator with the maximum value. The terminal elements are assigned minimum values. We have a representation class consisting of a vertex (the maximal element) and a pari of the expression corresponding to the lei) and right parts, if any, that are stored in a linked list that operates on these elements.
while (Estop)
{
sak = vvv. First;
left.ClearO;
right.Clear();
xxx.C'learQ;
for (int S = 0; i < 20; i++) if (sak. Value.GetX(i) != null) xxx.AddLast(sak.Value.GetX(i)); digit(xxx); f = 0; current = xxx.First; s = xxx.First.Value.GetNumber(); xx x. R emo ve F i rst(); node = xxx. First; f= node.Value.GetNumberO; if (xxx.Count() = 1)
left. Add Last( node. Val ue); else
while (node != null)
{
if(f<s)
left. Add Last( node. Val ue); if(f>s)
right. AddLastfnode. Value); f++; node = node.Next;
}
str = "";
if (current .Value .GetSy mbol() = "*")
str = "Mult"; i f (current. Value.GetSymbo l() = "/")
str = "Div"; if (current Vahie.GetSymbo l( ) = "+")
str = "Add"; if (current. Value.GetSymbol() = "-") str = "Subst";
if
(Char.lsDigit(Convcrt.ToChar(currentValue.GetSymbot())) = true) str = "Numb";
for (int i = 0; i < 5; i++) (
if (matrix[0, i] = str) {
str = str + Con vert .ToString(Con vert ,ToIut32(matrix[l,
'D+D;
matrixfl, i] =
C'onvert.ToString(Convert.ToIiit32(matrix[I, ij)+ I); break;
}
i
if (left.CountO > 0 && str.]ndexOt)"Numb")=-i) vvv.AddLast(new représentât i on(Str,
current. Value.GetSymbol(), left));
if (right. CountO > 0 && str.lndexOf("Numb")=-l ) vvv.AddLast(new représentai ion(str,
current.Value.GetSymbol(), right»;
ggg.AddLastfnew tree(sak.Value.GetPoint(),
sak.Value.GetSymboIe(), str, current.Value.GetSymbolO));
nedor.AddLast(new image(str,
current. Value.GetSymbolO));
vvv.Remove(sak);
if (vvv.Count() == 0) break;
}
Operators are replaced by vertices with appropriate notation. Links between vertices are displayed using contiguity lists. For greater clarity, expressions 1 ) 2*3+4*3-4/5; 2) 2*3+4/3+4/5; 3) 3*4+3*4-5/3; 4) 2+3+4+3; arc given shapes as follows: 1)
Mult 1 Add Substl N Limb I Mult! Numb2 Mult2 Substl Divl Numb3 Mult2 Numb4 Numb5 Divl Numb6 2)
Multl Add Add! Numb I Multl Numb2 Divl Addl Div2 Numb3 Divl Numb4 Numb5 Div2 Numb6 3)
Multl Add Substl Numb I Multl Numb2
7TT
Mult2 Substl Divi Numb3 Mult2 Numb4 Numb5 Divl Numb6 4)
Numb I Add Add 1 Numb2 Addl Add2 Numb3 Add2 Numb4
Even if the operators coincide or the leaves of the graph coincide, each of them is assigned a separate index, depending on the order of appearance in the graph.
At the next stage, optimization is performed to more cleariy visualize the tree to simplify comparison with the other binary graph.
There are two approaches. It seems that the search for a backbone tree using the Kruskal algorithm could be suitable for this (ask. This algorithm looks for the cheapest edges connecting the connected components of the graph, i.e. cycles. The algorithm operates on weighted graphs, determining in the first stage whether both endpoints of the candidate edge are in the same connected component. In the case of a positive test result, such an edge is discarded, because by adding it would create a loop in the future tree. If the endpoints are in different components, the edge receives and connects the two components into one. The structure of data storage is important. It is advisable to first group the vertices into connected components, and then search for the cheapest edges between the components.
while (node 1 !=null) {
currentl = nodel.Next;
while (currentl != null)
(
min = 1000; spot ="";
for (int i=0;i<nodel.Value.Length;i++)
lor tint j =0;j < currentl. Value. Length; j++)
{
node 1000 = bnm. First;
while (node 1000 !=null)
{
if (node I, Value.Substring(i.l) +
currentl. Value,Substring^, I) = node 1 OOO.Value.GetNameQ || currentl. Value. Subs tringfj, I) + nodel . Value. Substring(i,l) ==
nodelOOO.Value.GetNameO)
{
if (node 1000.Value.GetWeight() < min) {
spot = nodelOOO.Value.GetNameO; min = node 1000. Value.GetWeight();
}
}
node 1000 = nodel OOO.Next;
}
i
if (min != 1000) {
pal h. Add La st( s pot};
k= I;
break;
}
else
currentl = currentl .Next;
}
if(k=l)
i
sss.AddLast(nodel .Value + currentl .Value); sss.Remove(nodel); sss. Remove (current I); nodel = sss.First; if (sss.Count() = I) break;
}
else
nodel = nodel .Next;
}
In the arithmetic expression used as the model of the sentence, there are no cycles, therefore, there are no connected components, and there are only vertices with different degrees of their edges. In the approach to optimization, we consider the search for a vertex cover of a graph. The vertex cover does not coincide with the independent set, bccausc a vertex cover can contain vertices belonging to one edge.
We get the following expression optimizations:
1) Mult I SubStl Mult2 Divl
2) Multl Addl Divl Div2
3) Multl Substl Mult2 Divl
4) Addl Add Add2
As the method of comparing expressions, we use the remainder of sets intersection. The greatest coincidence is observed in the first and third expressions. The same approach can be used to compare the syntactic structure of sentences.
1. Sak A.N. (2015). ldcntificacia cliicnov prcdlojenia kak osnovnaia zadacha mashinnogo perevoda, I Identification of the sentence members as the main task of machine translation!, Moscow: Scientific Review Ed. 14. (in Russian)
2. Steven S. Skiena. (2011). Algoritmy. Rucovodstvo po razrabotke [The algorithm Design Manual], St,Petersburg, BHV. (/« Russian)
3. Paducheva E.V. (2009). Statyi raznyh let [Articles of different periods]. Moscow: Studia philological. 20 p.
4. Aiyoshintsev A. V.. Bessonova E.V., Sak A.N. (2017). Modelirovanie systemy maehinnogo perevoda technicheskih tekstov v telecommunicacionnoy sfere s ispolzovaniem objectno-orientirovannogo yazika programmirovania C# [Modeling of the machine translation system for technical texts of the telecommunication field using the object-oriented programming language C #] // T-Comm, Vol 11. No. 10, pp. 66-73. (in Russian)
5. Russel Stuart. (2006), Iskustvennyi intellect. Sovremennyi podhod.f Artificial intelligence. Modern approach]. Moscow/ St. Peters burg/Kiev.
6. Alyoshimsev A.V.. Jolodkov P.A. (2009). Obzor sovremennij tejnologiy ispolzuemij pri postroenii «ianiy [Overview of contemporary technologies used for smart-house design] II T-Comm. No. S2, pp. 159-163.
References
МОДЕЛИРОВАНИЕ ИНФОРМАЦИОННЫХ СИСТЕМ КОМПЬЮТЕРНОГО ЗРЕНИЯ ДЛЯ МАШИННОГО ПЕРЕВОДА МЕТОДАМИ ТЕОРИИ ГРАФОВ
Алешинцев Андрей Владимирович, Московский технический университет связи и информатики, Москва, Россия Сак Александр Николаевич, Московский государственный строительный университет, Москва, Россия
Аннотация
Компьютерное зрение научное направление в области искусственного интеллекта (робототехники в технологии получения изображений объектов реального мира, их обработки и использования данных для решения прикладных задач (без участия человека)). В информационных системах компьютерного зрения для области машинного перевода представление данных является основой (например, данных исходного предложения или связных компонентов изображения). Представление данных осуществимо с помощью графов, где в качестве вершин выступают слова, группы
пикселей или варианты решений, а в качестве рёбер - отношения между словами в предложении, элементами изображений или решений. В большинстве предложений отношение подчинения и линейного порядка связаны между собой. Представление синтаксической структуры предложения в виде дерева подчинения используется при построении порождающих грамматик языка и в алгоритмах синтаксического анализа. Дерево строят, отталкиваясь от распределения лексических единиц предложения по частям речи, затем осуществляется переход от дерева подчинения к дереву составляющих. Двоичное дерево поиска, является видом структуры данных, отвечающей представлению предложения в виде дерева составляющих.
Ключевые слова: компьютерное зрение, теория графов, дерево подчинения, дерево составляющих, бинарный граф, синтаксис, семантика, искусственный интеллект.
Литература
1. Сак А.Н. Идентификация членов предложения как основная задача машинного перевода. М.: Научное обозрение Вып.14, 2015. С. 9-14.
2. Стивен С. Скиена. Алгоритмы. Руководство по разработке. 2-е изд.: Пер. с английского.- СПб.: БВХ-Петербург, 2011. 720 с.
3. Падучева Е.В. Статьи разных лет. М.: Языки славянских культур, 2009. 736 с. (Studia philologica). 20 c.
4. Алешинцев А.В., Бессонова Е.В., Сак А.Н. Моделирование системы машинного перевода технических текстов в телекоммуникационной сфере с использованием объектно-ориентированного языка программирования С# // T-Comm: Телекоммуникации и транспорт. 2017. Том 11. №10. С. 66-73.
5. Стюарт Рассел, Питер Норвиг. Искусственный интеллект. Современный подход. Москва-Санкт-Петербург-Киев, 2006.
6. Алёшинцев А.В., Холодков П.А. Обзор основных технологий, используемых при построении "интеллектуального здания" // T-Comm: Телекоммуникации и транспорт. 2009. № S2. С. 159-163.
Информация об авторах:
Алешинцев Андрей Владимирович, старший преподаватель, Московский технический университет связи и информатики, Москва, Россия
Сак Александр Николаевич, к.филол.н., доцент, Московский государственный строительный университет, Москва, Россия
г I л
