CC BY
12
THE ARCHITECTURE OF ANDROID APPLICATION'S MODULE THAT TRANSMITS FRAGMENTED VIDEO RECORDINGS TO CLOUD SERVICES
K.V. Marchenko, student Novosibirsk state university (Russia, Novosibirsk)
Abstract. The paper overviews how the module that transmits video fragments to cloud services while the recording is being created works. The architecture of the module is considered and the main characteristics of the created video are described.
Keywords: mobile development, video data transmission, video fragments, cloud services, Android.
Nowadays, people constantly fix events by recording video. Recording is often done using mobile devices, the vast majority of which are the Android-based systems [1]. Video recordings not only can be stored on the device itself, but also transmitted via the Internet to remote servers during creation, and a convenient place for their storage are cloud services. Cloud services, however, do not support streaming media, so the problem arises: how to transfer video by fragments of small lengths so that they could later be assembled into a single file?
At first, recording module and its architecture should be considered.
The recording starts only when the user launches it. At the beginning, a video track is added, and after that video recording parameters are set: format of the recording is MPEG-4 Part 14 file, FPS is thirty frames per second and a time interval between keyframes constitutes one second. This means that every thirtieth frame is a key frame.
Next up, the actual recording is commenced. After receiving the next uncom-
pressed frame a check is performed to determine whether the frame is a key frame. If it is, this keyframe is saved into a buffer. The class instance responsible for video creation is then stopped, persisting the recorded video fragment to the local storage of the device. The observer that transmits the fragments is then informed that the recording has been halted. It is only then that the video creation class instance is resumed. If the frame was not a key frame, it is compressed using H.264 video compression standard. Afterwards the resulting frame is appended to the recording. In any case the next step is awaiting for the new frame.
After receiving the signal to stop the recording the current fragment is saved to the mobile device, while the transmitting component is notified about the end of the recording session. The recording itself is stopped shortly after.
Let us review the implementation of the component responsible for video recording. Its basic classes are presented on the class diagram on the following Figure.
Figure 1. Class diagram of the recording component
Launching, configuring and closing the camera is performed via CameraWrapper class. It is used to work with the camera hardware. CameraWrapper allows to start / stop camera preview and retrieve frames for coding to get a video recording.
In order to manage the recording and to save fragments periodically an instance of PeriodicFragmentRecorder class is created. In the constructor it is passed an instance of CameraGLView class, which is a subclass of GLSurfaceView required to render OpenGL content [2]. Recording launch is performed using the start method, which takes two parameters. The first parameter is the object containing the path for MediaMuxerWrapper [3] to save video recordings, while the second is the extension for the saved video fragmens. This
object also stores the number of the last fragment recorded by
MediaMuxerWrapper. Not only is this number useful for MediaMuxerWrapper itself to save fragments with correct numbers, but it is also used by PeriodicFragmentRecorder to notify the transmitting component about the saving of the next fragment.
An instance of MediaMuxerWrapper class, created by
PeriodicFragmentRecorder, allows to construct an MP4 file based on the encoded video frames. When the constructor of the class is called, it is passed an instance of VideoEncoder class, which handles video track encoding. On arrival of the next buffer with the data encoded by MediaEncoder, MediaMuxerWrapper ap-
pends it to the output container. Upon receiving the buffer containing a key frame MediMuxerWrapper notifies its SpecialFrameListener (which is also PeriodicFragmentRecorder) and passes is the buffer with a keyframe. After that SpecialFrameListener stops
MediaMuxerWrapper, by that also saving the fragment on the mobile device. SpecialFrameListener then notifies VideoFragmentListener about the succesful save of a fragment which makes it possible to transmit the fragment to a remote server. Following the notification, SpecialFrameListener restarts
MediaMuxerWrapper, passing it the keyframe buffer it received before. This keyframe becomes the first frame in the next fragment.
When PeriodicFragmentRecorder receives a signal to stop the recording (the user pressed the 'Stop' button), it invokes MediaMuxer Wrapper's stopRecording method, which in turn calls signalEndOfInputStream. This ends the reception of camera data in MediaEncoder, and MediaMuxerWrapper receives the buffer with an end-of-stream flag, effectively terminating. Before shutting down, MediaMuxerWrapper saves the last fragment even if it is less than a second long.
In the end SpecailFrameListener is informed about the end of the recording session, which passes that message to the transmitting system.
References
1. Mobile OS Market Share 2018 // Statista: The Statistics Portal. URL: https://www.statista.com/statistics/266136/global-market-share-held-by-smartphone-operating-systems (last access 5.06.2018).
2. OpenGL ES Overview / Khronos Group. URL: https://www.khronos.org/opengles (last access: 5.06.2018).
3. MediaMuxer / Android Developers. URL: https://developer.android.com/reference/android/media/MediaMuxer (last access: 5.06.2018).
A few words about transmission. The transmitting component initializes upon receiving a start notification from the recording component. The first step is to create a thread pool with four threads, where the load distribution is managed by a scheduler. All operations within the transmitting component are performed away from the main thread since the main thread is bound to UI updates and user interaction handling. If the main thread is busy with input/output requests and/or time-consuming network calls, the UI may stutter, or, in the more sever cases, the app might become completely unresponsive to all user interactions.
After the thread pool is created and passed to the job scheduler, a fragment is fetched from local storage. Once the fragment is retrieved it is sent to the server. Both operations are performed in a single background thread, which is then reused in the thread pool to upload further fragments.
Conclusion
The architecture of the universal component, ready to be built into any Android-based system with an API version greated than 21, is considered. The component allows to transmit fragments of video recording to cloud services right at the time of recording's creation, while the length of one fragment is reasonably small.
АРХИТЕКТУРА МОДУЛЯ ПЕРЕДАЧИ ФРАГМЕНТИРОВАННОЙ ВИДЕОЗАПИСИ В ОБЛАЧНЫЕ СЕРВИСЫ ANDROID-ПРИЛОЖЕНИЯ
К.В. Марченко, студент
Новосибирский государственный университет (Россия, г. Новосибирск)
Аннотация. В статье описывается, как работает модуль передачи видеозаписи по фрагментам во время её создания в облачные сервисы. Рассматривается архитектура модуля и описаны основные характеристики создаваемой видеозаписи.
Ключевые слова: мобильная разработка, передача видеоданных, видеофрагменты, облачные сервисы, Android.
