GAMMA HEIs

The five University Partners:- the Universities of Central Lancashire, Manchester, Lancaster, Salford and Liverpool (Including the Virtual Engineering Centre) have carried out internal research according to their allocated theme, including Application/Algorithm development, scientific literature surveys, feasibility studies, reports etc, to provide an understanding of technology gaps, identifying possible areas of focus for development and exploration in the project and to provide an understanding of user requirements for selected applications.

When required, HEIs have performed Application /Algorithm development in their specialist area to support specific technology areas within SME applications.

GPS/GNSS Simulation (University of Liverpool)

The University of Liverpool has created a GPS simulation that bridges a synthetic environment to real world physical hardware. The synthetic environment supports a DIS (Distributed Interactive Simulation IEEE1278) network interface, and the hardware utilises a RS232 serial link sending NMEA0183 messages (National Marine Electronics Association 0183) which is a commonly supported GPS standard. Internally, the GPS model uses a satellite constellation model to calculate the visibility of actual satellites given a recent RINEX Navigation File (Receiver INdependent EXchange Format).

Wide Field Of View senor simulation (ARGUS-IS simulator) [University of Liverpool]

Wide Field of View sensor systems, allow persistent, continuous video surveillance of an entire town or city (approximately a 6km diameter area on the ground). These systems produce vast quantities of image data, that needs to be reduced down to metadata and regions of interest before transmission. Simulation of such as sensor allows it to be quickly positioned in a scene without the need to access and fly the sensor. Ground traffic can be controlled and suspicious vehicles added. The University of Liverpool can generate visually consistent and detailed Wide Field-of-View imagery using one or more instances of a COTS (Commercial Off The Shelf) flight simulator, and a traffic simulator to create thousands of persistent targets in the world. Approximating the specifications of the BAe/DARPA ARGUS-IS sensor.

RGB-D Model (Synthetic Kinect) [University of Liverpool]

The University of Liverpool investigated the use of COTS (Commercial Off The Shelf) gaming software as an environment to implement a real time simulation of a RGB-D (Red,Green, Blue & Depth) sensor. The sensor model operates at 30 frames per second, and approximates the popular Microsoft Kinect sensor (Version 1). The system could be used for simulating point cloud information for UAVs (and Ground Robots). This allows testing without the need to fabricate and store complex scenes, and likewise the camera view can be set to any position without the need of a platform or rig.

Pathfinder (University of Manchester)

This Pathfinder app demonstrates the procedural phases involved in the development and deployment of apps within the GAMMA program. It highlights the process of transforming an idea or sets of codes into an app using the GAMMA architecture.

The app focuses on pattern recognition and tracking of a moving object; it is effectively a payload system that detects and tracks an OoI (a red rectangle, which is an approximation of a red car) using a hybrid colour/shape detection algorithm. In essence, the app can simultaneously distinguish between multiple colours and shapes. The app can detect an OoI at a max distance of 1.96m which can be scaled to represent a car at an attitude of 78.4m, in an operating range of 60-400Lux. It can track OoI movements at a distance of 2.0m, at a max speed of 1.1m/s, and resolve the coordinates of the OoI in approximately 0.10s. A combination of algorithms is used to stabilise the video feed, with the capability of reducing perturbations by 60% at a displacement of 10mm, operating at frequency up to 8.5Hz. Hardware stabilization is implemented by combining motion sensor data with the controller in order to maintain line-of-sight with the OoI – roll axis perturbations of up to ±23°can be resolved parallel to the horizon in less than 0.5s. A packet pair/packet sniffer application is developed to dynamically measure bandwidth usage of the network for the purpose of prioritising data transmission, which reduced bandwidth usage by up to 70% when combined with a blurring feature. Hardware & software stabilisation, object detection, gimbal control and telemetry sub-systems are all integrated with the ARMFlash platform that has been developed in-house. The app comes with a GUI that has full control/access to the entire system.

Multi-Sensor Fusion (University of Manchester)

In accordance with the central theme of Growing Autonomous Mission Management Applications (GAMMA) project, the University of Manchester is primed to provide a platform that facilitates sensor integration, while supporting the entire GAMMA programme with a rich repertoire of knowledge on every aspect of the sensor and control theme. Part of the task is also to provide a laboratory based set of sensors and sensor control systems for the support of developmental work within GAMMA. Therefore in this project, a laboratory based demonstration of multiple sensors fusion is showcased for the project partners and stakeholders within the GAMMA programme who are interested in developing advanced apps that use multiple sensors possibly from different platforms.

The University of Manchester, particularly School of EEE, has developed the mission management hardware board, namely RedCape to continuously provide the world-class expertise and support in UAV multisensor integration and control. It can be observed that RedCape with selectable sensors is capable of being integrated in the UAV platform due to its lightness as well as suitable size to execute multiple tasks. With this board, a lab-based demonstration, orientation tracking of a moving object by fusing the information of accelerometer, gyroscope as well as magnetometer integrated in the RedCape mission management board, is developed in this project. System structure is given as follows:

Expected results have shown the feasibility of multisensor fusion in a lab-based platform and it can be anticipated to be extended to the UAV/UGV platform. Also it can be anticipated that more sensors can be fused in the framework to obtain a more accurate result. This effort is to encourage and stimulate project partners and stakeholders within the GAMMA programme of particular interest in using multiple sensors in UAV platform. UoM will keep committing itself in GAMMA project to provide world-class expertise in the area of sensing and control.

auROS (University of Manchester)

AUROS is a collection of software which simplifies writing software for drones. A large part of any software application developed to run on an aircraft is the ‘plumbing’ – no innovation, no IP, but it’s required in order for the ‘core’ of the software to properly function. In some cases, the amount of plumbing may greatly exceed the unique differentiator of the software; but sucks resources away from other development tasks.

With AUROS, the plumbing is already written and maintained. A basic image processing application can be written in a few lines of code. AUROS supports the programming languages C, C++ and Python, the majority of languages used for writing embedded software.

Existing software can also be ported to AUROS; an internal example application went from ~250 to 10 lines of code. Another advantage is that AUROS applications are network transparent; they can be run either on a ground based computer or aircraft based unit with no modifications. AUROS also provides compatibility across devices; apps are not tied to a certain brand of camera – getting images from a colour camera for a thermal imager requires no code changes.

Technically, AUROS is a collection of ROS (Robotic Operating System) packages particularly useful for unmanned aircraft use. The ROS project is the premier open source robotics middleware system, used in both commercial products and academia across the world. The BSD license of ROS makes it simple for companies to incorporate it into their products without IP issues. The University of Manchester provides precompiled, signed & packaged versions of the software, fixes issues in conjunction with upstream developers, writes and maintains AUROS drivers for specific hardware and provides examples and installation documentation. For SMEs who prefer more control over their stack, AUROS also specifies a particular linux environment and the packages available. AUROS is completely open source, so anyone is able to suggest additional packages they would like made available, improve the documentation or fix bugs in the software.

AUROS:

Greatly reduces development time & risk
Enables platform independence across airframes & sensors
Promotes good software engineering methods
Provides samples and documentation

Acts as an informal interface contract for SMEs lacking verified software development experience

Human Computer Interface for Remote surveillance Applications (University of Central Lancashire)

This gamma application created a human computer interface designed for remote surveillance tasks. The developed system uses a range sensor to estimate the operator head position and orientation in real-time. The estimation is based on a random forest’s head detection and pose regression model with an integration of the Kalman ﬁlter. The remote scene is captured by a 3 degrees of freedom robotic camera head and the video is showed to the operator on a head mounted display to increase the situation awareness. The developed non-contact head tracking system is used to control the robotic camera by replicating measured operator’s head motion.

A head pose tracking system for immersive applications has been developed. The method has been explained in a paper submitted to the International Conference on Image Processing. The spatial immersion is improved by adding a 3 degrees of freedom (3-DOF) camera to the system. The proposed framework is person independent and can work with diﬀerent models of HMD. Using low cost range data sensor oﬀers several advantages, such as an inertial unit is not needed and the control of other instruments could be added without using new hardware. For instance, tracking the hands movements to modify vehicle parameters. The future work would consider using the head movements to control the camera zoom with a natural gesture of the head. The designed system is universal in the sense of that can be used for detection and tracking of diﬀerent objects in range data stream, as long as there is suitably annotated data set for the random forest training.

Automatic Post Processing Algorithm (University of Lancaster)

We developed computationally efficient methods and algorithms for video analytics. Although developed methods done offline and post-flight they reduced the human analysts considerably and mostly used for decision making and not frame by frame manual analysis

Developed algorithms by Lancaster University can be categorised into two scenarios.

In the first scenario, the developed approach is fully autonomous (no human involvement is needed) replacing the need of human operators for video analytics tasks. It is the template-based approach includes matching an image/frame with another image/frame, taken a priory multiple templates from different views/angle. In this method, a catalogue/library of images of expected object(s) is collected and available to the algorithm. The proposed technique is then automatically detects an object that is visually similar to any of the images from the catalogue/library of templates.

In the second scenario, the method detects and tracks unknown object(s) autonomously. The important difference in this method compared to the previous ones is that any moving objects is automatically identified and clustered.

All methods have been developed in C++ environment and tested on videos recorded by AR Drone 2.0 at Lancaster University and UAV footage provided by Manchester University. All Demos have been uploaded into eRoom.

Cloud Mapping of Unknown Environment (University of Salford);

Autonomous Navigation and Collision Avoidance (University of Salford);

Autonomous Navigation and Motion Planning (University of Salford)

Report 1 presents the application of swarm based mapping within the context of cloud. In this experiment, two UGVs are tasked to map a large portion of a building and they upload their maps in to the cloud while carrying the task. The maps are merged in the cloud and a complimentary map is generated in real time.

Report 2 utilizes fuzzy logic approach for obstacle avoidance and autonomous navigation of a single UGV (turtlebot) in static and dynamic environments

Swarm Intelligence based Algorithm (University of Salford)

It is the decision of our development and academic team members that the current version of the report requires further work and investigation before it can be released. This was planned to be finalized by October however due to allocating extensive time and man power to SME support (InTouch app 055 and 059) this is now slipped to December.

M. N. Ab Wahab & A. Atyabi & S. Nefti-Meziani, Application of macroscopic modelling of PSO on robot motion planning. The report is targeted for publication in IEEE Robotics and Automation Magazine (RAM), impact factor 2.484, or any other journal with similar reputation and impact. The paper offers a comparison among conventional (Dijkstra and potential field) and swarm based (GA, PSO, DE) navigation approaches and has some synergies with the overall theme of report 7 in the sense of examined SI based approaches.

The results indicate the advantage of macroscopic modelling of PSO in comparison with other SI based methods and PF. In addition, the statistical analysis of the results indicated that macroscopic model of PSO is capable of performing on pair with Dijkstra algorithm while the former has the advantage of having complete knowledge of the environment and the location of obstacles in contrast to PSO that only has likelihood information of starting and goal positions in the search space.

Search and Rescue Drone (University of Salford)

Activity resulted in a low TRL (e.g., TRL 2 feasibility study) algorithm development capturing the use of Optical Flow method for obstacle avoidance. The remaining of the work is suspended due to the lack of required resources and funding. The results are planned to be release to Gamma as report 9 in Dec.