ID:

SW-1

Embedded

SUPERVISOR:

Simon Winberg

TITLE:

Real-time Emulation Framework for Linux (industry linked!)

DESCRIPTION:

In many applications it is desirable to have the guarantees offered by a real-time system as well as the convenience of working in a Linux system. Although Linux isn't a real-time OS, techniques can be used to approximate one.

The objective of this project is to investigate what existing facilities offered by Linux could make it behave more like a real-time system, for example using real-time priority levels, kernels space programs and dedicating cores to real-time tasks. Characterise the trade-offs of each approach utilized. From these findings, construct an application framework that will make it easier for developers to utilize your RT emulation techniques to develop their own programs.

A suitable case study needs to be constructed to trail the techniques in a real-world real-time application. A suggestion is sound source localization or sonar.

At least the following metrics should be considered in determining the behaviour and performance of the system: throughput, average latency, maximum latency, latency jitter, wasted resources, and possible overhead.

DELIVERABLES:

Application framework for RT Emulator Framework for Linux, in addition to sample code (case study) showing how to use the framework. Student doing this project is expected to report regularly to the supervisor (S. Winberg) and to provide regular fortnightly brief progress updates to our industry partner. Two demos (one a term) are planned to show the industry partner concrete results of progress that is being made.

SKILLS/ REQUIREMENTS:

Embedded Systems / Software Engineering

ELO1: Problem solving: Identify, formulate, analyse and solve complex* engineering problems creatively and innovatively

A crucial property of real-time performance is the predictability of the timing of the system. Standard Linux distribution do not, for example, guarantee the latency in responding to an interrupt or the maximum latency of a context switch. As part of the problem-solving strategy for this project, the student needs to consider likely RTOS requirements and how these can be emulated, limited to available Linux kernel routines and user-level code. Part of the problem is the desirability of the framework not to be restricted to particular distributions of Linux; restrictions could be made to a kernel version – but ideally the framework could be ported between kernel versions as well. But, due to time and complexity constraints of this project, the extent of how portable the framework should be, and how extensively this portability will be tested, needs to be determined. It probably only be possible to provide emulation of only a small portion of the full set of RTOS characteristics.

ELO 4**: Investigations, experiments and analysis: Demonstrate competence to design and conduct investigations and experiments.

The problem presented here is fairly open-ended in terms of the RTOS features to emulate, but those features that are to be implemented need to be thoroughly designed, tested experimentally and the results thoroughly analysed to determine to what extent the operational characteristics have been achieved. Thorough experiment planning will be needed, and it is recommended to formulate a series of experiments to validate performance, and to incrementally test that as more RT emulated services are added to an application, the individual performance characteristics of these services are not overly degraded when increasing number of these features are combined.

AREA:

Computer Engineering

ID:

SW-2

SUPERVISOR:

Simon Winberg

TITLE:

Diagnostic Reporting Framework for NN Predictions

DESCRIPTION:

The Scientific Computing Research Unit (SCRU) at UCT makes use of various Machine Learning (ML) methods, such as artificial neural network (ANNs), self-organizing maps (SOMs) and deep learning to develop applications for early prediction of various types of disease from data obtained from tissue and blood samples. The focus is at present towards early prediction of different types of cancer.

Problem overview: We have a team of students developing an application to facilitate early detection and diagnosis. But a common problem with many of these methods, such as ANNs, is that they don’t provide reasoning or motivation for the predictions that they give. Indeed, this can’t really be expected for a standard multilevel feed-forward perceptron-type structure. However, adding a facility to help establish the reason for predictions would have many advantages. For example, false diagnostics can be obtained by artefacts in the input data skewing predictions.

Objective: what we are wanting is a reason together with a prediction to be generated by a ML prediction method. It is suggested that the prediction mechanism is provided by a multilayer neural network (however, the student may decide a different predictor e.g. unsupervised learning methods, such as a self-organizing map and the diagnostic report concerning reasons for a classification given to an input).

Literature considerations: The student should consider, as part of the literature review, investigation of interpretability methods, such as used by Grad CAM[2], LIME[3], Attention[4], PCA, feature visualisation [5] and Bayesian networks [6].

Input and outputs: Alternate types of inputs can be tested; the student can explore these options as part of the lit review. A suggestion is investigating generating diagnosis reports for interpreting medical images. The reasoning for using these images are high important, for various reasons e.g. as indicating the type or severity of the illness predicted. Furthermore, the impact of trading-off false positives and false negatives could be assessed; in certain cancer illnesses, for instance, the cost of a false positive – be it monitory or emotional – may be very high particularly if the diagnostic was erroneous. For this reason, means by which automatic diagnostics or predictions can provide more substantive evidence, which a human may be able to review and interpret, would be of much benefit to this field.

Workplan: It has been proposed that this project be done in collaboration with the Scientific Computing Research Unit (SCRU) at UCT who are actively engaged in researching machine learning methods and methods for diagnosing cancer illnesses using medical images (for this project ‘fake’ or open access images are recommended for use to simplify ethics approval). For this project you would be encouraged to work in the SCRU lab with the postgrad students doing ML, and use the group’s high-performance server.

Optional extra: If time permits it is desirable to provide a visualization for the reporting results, if a visual report of some sort is done. But with the visual there should also be a text-based report. Visualizations such as a U-matrices or scatter plot with e.g. overlaid labelling for prediction reasons could be done. The SCRU lab has a ‘vision-wall’ that you could try out as a means to present your visualizations. You could allow for additional UI and visualization controls to allow for adjusting parameters when running predictions or during the training of the NN.

This topic could lead on to an MSc related to Scientific Computing / Machine Learning with the SCRU group (postgraduate funding available).

DELIVERABLES:

The implementation of a training and testing system for the deep learning application.

A comprehensive performance evaluation of at least one solution method.

SKILLS/REQUIREMENTS:

Good math skills. Software Engineering. Programming skills.

(You don’t need prior experience with machine learning/NN, an interest in algorithms is desirable).

ELO1: Problem solving: Identify, formulate, analyse and solve complex* engineering problems creatively and innovatively

The student will have to formulate an approach based on available resources and literature and develop a proposed solution to the problem. The topic specification is vague, so different options will have to be considered and compared in order to identify the problem. The range of implementation options is vast and there is complexity in both the selection of a method to implement and in the execution thereof, as part of the project the student will need to refine the scope of the solution and decide effective measurement techniques to assess the accuracy and performance of the prototyped system.

ELO 4**: Investigations, experiments and analysis: Demonstrate competence to design and conduct investigations and experiments.

The performance of the proposed algorithms will have to be qualitatively and quantitatively evaluated. This will involve formulating experiments to test proposed hypotheses and drawing appropriate conclusions from generated interpretations.

EXTRA INFORMATION:

Done in partnership with SCRU: http://www.scientificomputing.uct.ac.za/ (workspace will be provided in computing lab close to vision wall)

References:

[1] Guo, Wenbo, et al. "Explaining Deep Learning Models--A Bayesian Non-parametric Approach." Advances in Neural Information Processing Systems. 2018.

[2] - https://arxiv.org/abs/1610.02391

[3] - https://arxiv.org/abs/1602.04938

[4] - https://arxiv.org/abs/1706.03762

[5] - Feature Visualization, C. Olah, A. Mordvintsev, L. Schubert. Distill. 2017.

DOI: 10.23915/distill.00007; https://distill.pub/2018/building-blocks/

[6] - https://dash.harvard.edu/bitstream/handle/1/33840728/KRAKOVNA-DISSERTATION-2016.pdf

AREA:

Computer Engineering / Software / Machine Learning

Project suitable for ME/ ECE/ EE?

ECE

RESERVATION

Not reserved

ID:

SW-3

Embedded

SUPERVISOR:

Simon Winberg & Dr Lerato Mohapi

TITLE:

Integration of Smart Personal Devices for Security and Health Monitoring in Cars (HMC)

DESCRIPTION:

This project is about the design of an integrated smart personal security and health monitoring system. This system is aimed at providing a multi-input automotive security system that cross-checks the individual’s identity based on biometric information, such as the following:

• Driver's car Seat-based Weight Sensor (SWS)
• Visual recognition (i.e. facial)
• Steering Wheel Finger printer reader (off the shelf device)

When output from majority of above sensors is ‘yes’, that it is the right person, then the car ignition can automatically or manually start, otherwise it locks itself, while sending a message to the owner’s cell phone to

allow use (i.e. by entering an OTP), or if the owner wants to alert the police or security automatically. Furthermore, these sensors can be extended by introducing the following sensors for additional health monitoring:

• Seatbelt heart rate monitor
• Seat Angle Monitor for monitoring safe driving seat angles
• Visual analysis of facial expression to determine if long distance drivers need some rest, etc.

When the person's heart rate is abnormal, the system must alert the driver. When overweight and/or gaining significant amount of weight, the system must alert the user as this may affect the security knowledge-base and his health. When the facial expression analysis provide signs of sleepiness, the

system must alert the driver as this may cause accidents.

We of course do not expect all these features to be added, but this project can rather be a starting point. A major focus, and which could be the main deliverable due to time limits, would be the development and testing of the driver’s seat-based weight measurement sensor.

DELIVERABLES:

- Design of this smart personal system

- Example devices which can be used for this systems

- Security test case: Integration of Weight sensor system with

facial and finger print recognizer using Arduino YUN board and

a software based facial and fi nger print recognizer

- A USB camera can be used to mimic real-time facial

recognition, but finger pints can be pre-recorded.

SKILLS/ REQUIREMENTS:

- Programming using C/C++

- Image processing techniques

ELO1: Problem solving: Identify, formulate, analyse and solve complex* engineering problems creatively and innovatively

The student is recommended to start with an initial literature review to become more familiar with available technologies of personal health monitoring and securing in motor vehicle. Thereafter the scoping of the investigation into the specific health monitoring device and its constituent components needs to be explored, this involves problem solving in terms of determining low-cost components and/or development kits suited to the system prototyping and prototyped system testing that will be completed in this project. This will likely involve deciding a series of trade-offs of choosing inexpensive components that will still provide accuracy and performance of sensing useful to this application.

ELO 4**: Investigations, experiments and analysis: Demonstrate competence to design and conduct investigations and experiments.

An testing rig, or experimental setup, of some sort would need to be constructed for this system. This may involve placing sensors in (e.g.) a office chair to do lab-based testing, and then transferring this setup to a (stationary) car in order to do authentic testing of the system in the target context.

Experiments will need to be designed and the data capture process planned so as to formulate an approach to carry out the tests using a limited budget and limited resources. It is expected that signal processing algorithms will need to be developed or adapted for this application. The performance of these algorithms should also be analysed and performance testing on embedded platforms carried out.

AREA:

Computer Engineering / Image processing / Software Engineering

RESERVATION

Not reserved

ID:

SW-4

Embedded

SUPERVISOR:

Simon Winberg

TITLE:

ActionTracer Sensor and Android App API

DESCRIPTION:

This project is about designing a small electronic sensing device that can record motion or vibration and can either log the motion internally or transmit the motion via a short wireless link to a receiver unit (a cell-phone) that will log and display information about the action.

This application involves both the development of the ActionTracer Sensor and the Android code for the ActionTracer App. Each of these aspects are detailed further below. The illustration below shows how the action tracer might be used. Note that the application doesn’t have to be golf: you could choose an alternate sport, such as darts, rowing or speed-walking. In some cases, it may make more sense to have the ActionTracer attached to the person (e.g. the speedwalker may wear the device around an ankle, but the rower might have it on an oar).

It is important that the ActionTracer sensor be small and battery-powered. Ideally last about 4h (but understandably, this may be hard to achieve for a first prototype).

The Android App is the user interface and data logger. It needs to be designed for  customizability, to integrate with a larger (e.g. future cloud-based) application e.g. as the diagram below suggest to app that logs golf swings and show statistics about the swings.

As indicated in the picture, the sensor should be small enough to connect as needed – you can decide to focus on a specific sport, e.g. gold and decide the size constrains accordingly. The Android App code should be customizable to provide useful logs or displays about the activity being pursued. The golf application could for example be implemented more as a case study for using the ActionSensor for recording information about golf swings. This could be done quite simply in the lab using a metal rod to substitute for a golf club – although ideally it should be tested on a real golf club to provide input in its appreciations in such an application.

If you’d prefer measuring activity concerning a vehicle or driver, please consider “ComfyRide App” or “Health Monitoring in Cars (HMC)”).

Design considerations:

Action Tracer Sensor: the sensor needs a tri-axis accelerometers to measure motion in pitch, roll and yaw. A small microcontroller, e.g. an 8-bit PIC, needs to be constantly reading the motion angles. For power-saving the system could possibly be design such the microcontroller is in sleep mode while there is no motion, and a purely analogue ‘sentry’ circuit wakes up the microcontroller when movement is detected. The user could perhaps know that the first motion will not be fully recorded (but for the golf app example this should be no problem because the club will be moved around for many seconds before a swing is attempted). The sensor can send out a datasteam over Bluetooth or similar low-power PAN wireless link.

Action Tracer Sensor App API: at a minimum the host software side of this project just needs to receive the wireless motion data stream and record this into a file (e.g. a csv file). It is imperative that the provided code is designed around being a reusable API!! i.e., not a once-off application – it needs to be program code that can become the baseline for further software that can implement specific applications using the wireless link and motion log. But preferably for this project a case study should be developed showing how the API code can be reused to e.g. develop the golf swing statistics application illustrated above.

DELIVERABLES:

At least one rugged and operational ActionSensor Tracer able to send motion sensor logs wireless. API code that can receive the wirelessly transmitted motion sensor logs and provide a means to show this information (e.g. on the screen of an Android device). Demonstration.

SKILLS/ REQUIREMENTS:

Embedded systems, Android development desirable (but not essential, can implement the host using a PC)

ELO1: Problem solving: Identify, formulate, analyse and solve complex* engineering problems creatively and innovatively

The student should begin with a literature review to become familiar with technologies and components suited to this application. Scoping of the investigation, in order to focus the problem-solving investigations, are needed to decide the types of sports that this system would accommodate and one (or a limited number) of sports that this system would be tested on. A series of trade-offs is likely needed in deciding the types of motions that the ActionSensor would be sensitive to, these decisions may result in constraining the types of sports that this system can be used with. Part of the problem solving is deciding design components to use, as well as deciding both testing artefacts and testing contexts to make use of in experiments.

ELO 4**: Investigations, experiments and analysis: Demonstrate competence to design and conduct investigations and experiments.

A comprehensive set of experiments needs to be planned, ranging from initial testing of a partially complete system (e.g. individual sensor tests) towards cumulating in full system testing within a sporting context (e.g. getting a professional golfer to use the system on a driving range). For the purposes of frequent lab-based testing, a suitable test rig may need to be constructed in the lab. For all experiments, the data capture process needs to be well planned to ensure good data is obtained for the report.

As part of this project, it is expected that a variety of signal processing algorithms will need to be developed or adapted. This will likely necessitate a level of software testing to examine the performance and memory costs of these algorithms and their applicability to resource constrained platforms.

AREA:

Computer Engineering  (wearable computer, embedded systems and signal processing)

ID:

SW-5

Embedded

SUPERVISOR:

Simon Winberg

TITLE:

Dance Cuffs

DESCRIPTION:

This system is planned to help people learn new dance moves. Training how to do a new dance move, by looking e.g. at a video, can help the learning to some extent to know how the different moves are to be done. But being able to remember the sequence effectively, especially of you don't have the video to remind you, can be quite tricky! Furthermore, there's a certain delay between seeing a movement on the video and linking this to the right part of the body to move – typically there's a lag, maybe it’s the rhythm learning curve, in which you for example see the performer in the video move their left arm (which may actually be your right arm if they are facing you) and you need a little time to ‘process’ which part to move. It could mean a bit of delays, indeed you might have to see how sequences multiple times before actually trying the moves out. But that’s were it’s hoped the “Dance Cuff” could speed things up. The concept of the dance cuffs and app is to have a means by which the cuff provides vibrations, or some sort of key that the user can feel, that helps them know what needs to be done. Perhaps it would be cuffs needed just for the wrists,

maybe for the legs as well; part of the investigation would be deciding where the cuffs or triggering devices need to be placed. Perhaps having just one cuff on one hand would be enough - eg. if it's has two buzzers maybe having the one is associated with the right, the other with the left even thought it's just on one hand. It's largely an investigation to see if this type of product can help a user learn new types of complex movements, and to be able to be reminded of these movements, through this form of device that uses non-vision sensory input the uses for this reminding and training.

The app could have a mode for training new movements, and would ideally then have a procedure whereby the movements can be linked to a video, e.g. something like a midi editor for synchronizing, or editing the start/end, of movements to link with points of a video. But going that far with the app is not anticipated, the “moves” can all be manually encoded e.g. in csv file which indicates the type of activation and duration. There would be a means to start the dance cuff and start the video/music (perhaps it’s simply just pressing to buttons at the same time to synchronize things).

Design considerations: In real-time applications, as this one probably would be, it is desirable to have the guarantees and timing analysis offered discussed in the RT system design. Possibly a O/S such as freeRTOS would be used. This allows lots of scope for real-time embedded design documentation, which will be an expected significant component of the report. 

DELIVERABLES:

Prototyped Dance Cuff system. Detailed design. System testing and analysis. Student should consider using volunteers to test the device and fill out a short survey concerning their impressions of it. The Student doing this project is expected to report regularly to the supervisor (S. Winberg), providing at least regular fortnightly brief progress updates. Two demos (one a term) are recommended as an opportunity to receive detailed feedback from the supervisor.

SKILLS/ REQUIREMENTS:

Embedded Systems / Software Engineering

ELO1: Problem solving: Identify, formulate, analyse and solve complex* engineering problems creatively and innovatively

The student should begin with looking at some training videos, a type of dance style can be focus on, e.g. Latin dance. The problem space would then need to be explored, e.g. the types of dance motions, the steps and pace, music used in training etc would need to be investigated to clarify the problem space. The types of motion sensors, enclosure and communication strategies for development of the Dance Cuff needs to formulated that will lead to determine a parts list and possible development of prototype pcb for placement and connection of these parts.  A series of trade-offs may be needed in this problem-solving exploration, to choose parts that are sufficiently affordable and low-power. Part of the problem solving is also deciding the software needs, any hardware-software interfacing needs, as well as determining algorithms that may need to be constructed for cueing the user’s dance moves and processing the samples in order to analyse and if necessary generate response cues to correct movements the user makes.

ELO 4**: Investigations, experiments and analysis: Demonstrate competence to design and conduct investigations and experiments.

A comprehensive set of experiments needs to be planned, ranging from initial testing of a partially complete system (e.g. individual sensor tests) towards cumulating in full system testing within a suitable use context (e.g. in a dance hall). For the purposes of frequent lab-based testing, a suitable test rig may need to be constructed, e.g. parts of a mannequin that can be easily moved to imitate movements of a real person, but for which alternate sensors or measurement tools can be rigged up to compare the accuracy of the Dance Cuff sensors to accurate measurements (e.g. the motion may be manually measured by rules and protractors and the motion plotted on paper and then compared to measurements made by the device). For all experiments, the data capture process needs to be well planned to ensure good data is obtained for the report.

A variety of algorithms tests will likely be needed, and it is recommendable to test these (e.g. using simulated data from MATLAB), in addition to testing these algorithms running on the platform being constructed.

AREA:

Computer Engineering  (wearable computer, embedded systems and signal processing)