Hindawi
Journal of Engineering
Volume 2020, Article ID 3645729, 16 pages
https://doi.org/10.1155/2020/3645729
Research Article
Hardware Module Design and Software Implementation of
Multisensor Fire Detection and Notification System Using Fuzzy
Logic and Convolutional Neural Networks (CNNs)
Robert A. Sowah ,1 Kwaku Apeadu,1 Francis Gatsi,2 Kwame O. Ampadu,1
and Baffour S. Mensah1
1Department of Computer Engineering, University of Ghana, P.O. Box LG 77, Legon, Accra, Ghana
2Faculty of Engineering and Cmputer Science, Ashesi University, Berekuso, Eastern Region, Accra, Ghana
Correspondence should be addressed to Robert A. Sowah; rasowah@ug.edu.gh
Received 16 September 2019; Revised 20 November 2019; Accepted 8 January 2020; Published 1 February 2020
Academic Editor: Franca Giannini
Copyright © 2020 Robert A. Sowah et al. )is is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is
properly cited.
)is paper presents the design and development of a fuzzy logic-based multisensor fire detection and a web-based notification
system with trained convolutional neural networks for both proximity and wide-area fire detection. Until recently, most
consumer-grade fire detection systems relied solely on smoke detectors. )ese offer limited protection due to the type of fire
present and the detection technology at use. To solve this problem, we present a multisensor data fusion with convolutional neural
network (CNN) fire detection and notification technology. Convolutional Neural Networks are mainstream methods of deep
learning due to their ability to perform feature extraction and classification in the same architecture. )e system is designed to
enable early detection of fire in residential, commercial, and industrial environments by using multiple fire signatures such as
flames, smoke, and heat. )e incorporation of the convolutional neural networks enables broader coverage of the area of interest,
using visuals from surveillance cameras. With access granted to the web-based system, the fire and rescue crew gets notified in
real-time with location information. )e efficiency of the fire detection and notification system employed by standard fire
detectors and the multisensor remote-based notification approach adopted in this paper showed significant improvements with
timely fire detection, alerting, and response time for firefighting. )e final experimental and performance evaluation results
showed that the accuracy rate of CNN was 94% and that of the fuzzy logic unit is 90%.
1. Introduction lost due to fire outbreaks in the first quarter of 2012 in Ghana
[2].
Fires have a duality such that while they can be useful in Fire safety experts say that when there is a fire outbreak
cooking and manufacturing; they can also cause loss of lives in the home, workplace, or factory, the occupants usually
and destruction of property. With fire and some of its by- have just about two minutes to escape before it is too late.
products being employed in a lot of useful applications such Fire-related loss of lives and properties continue to increase
as cooking, power generation, and manufacturing process, despite vigorous fire safety campaigns being carried out by
among others, it has the potential also to cause havoc. the public agencies. In developing countries, typical fire
According to the Center of Fire Statistics of the Inter- detection relies on humans. In cases of distress, poor town
national Technical Committee for the Prevention and Ex- and country planning make it difficult for firefighters to
tinction of Fire (CTIF), between 70,000 and 80,000 deaths arrive on time. )e problem is further compounded by
occur annually due to fire outbreaks since the beginning of inadequate alert and notification mechanisms. Fires are,
the 21st century [1]. In fact, GHC 1.74M (0.92M USD) was however, so destructive that they can annihilate a whole
2 Journal of Engineering
fortune within hours. )erefore, this situation must be multiple sensors, thereby making it easier to analyse the
tackled. It is of utmost importance that fire outbreaks be often-noisy sensor data. Fuzzy logic also best approximates
detected and that occupants and the fire service be alerted the actual values of the various fire signatures observed for
within a very short time. accurate and efficient detection.
)ere are three main elements required for fire to exist; )is paper is organized into sections. Section 1 provides
these are oxygen, heat, and fuel. )ese make up what is the introduction and necessary background for multisensor
known as the fire triangle. )e proportion of each of these fire detection using fuzzy logic and CNNs. It presents the
elements determines the nature of the fire. Before the in- background and the motivation for the research work and
troduction of technology, fire detection was done mainly by the problem statement. Section 2 presents the literature
visual inspection and confirmation. )is, however, changed review on fire detection systems. It highlights the various
during the late 1930s when Walter Jaeger accidentally dis- techniques for fire detection based on the use of sensors for
covered a sensor that can detect smoke, thereby paving the sensing fire signatures and video sequence processing for
way for research into modern smoke sensors. Until recently, fires. It identifies the strengths and weaknesses of such
most consumer-grade fire detection systems relied solely on systems and proposes a novel method using fuzzy logic and
smoke detectors. )e protection provided by these fire CNNs. Section 3 provides the novelty of the system design
detectors is limited by the type of fire present and the de- architecture and methodology, including the concepts, al-
tection technology in use. )e need to produce a more gorithms, and flowcharts. It highlights the various modules
reliable fire detection system devoid of false alarms has led to with their design and procedures for implementation.
the adoption of multisensor approaches. Unlike fire detec- Section 4 focuses on the actual implementations with ex-
tion approaches relying solely on smoke, this approach relies perimental verification and corresponding integration of the
on the detection of more than one fire signature. An developed modules. )is section provides sufficient details
emerging phenomenon in fire detection is the use of mul- on the testing and performance evaluation of the proposed
tiple parameters and sensors in the detection mechanism system. Finally, Section 5 provides conclusions on the
[3, 4]. Somemultisensor fire detection techniques being used system design and methodology and makes relevant rec-
currently include data fusion algorithms [5, 6]. )ese ommendations for future enhancements.
techniques allow different fire signatures to be captured and
processed together for ultimate fire detection. 2. Literature Review
Similarly, image processing techniques with neural
networks are employed in [7–9] for fire detection. )ese Sowah et al. [13–16] proposed the design and imple-
algorithms use image processing techniques with neural mentation of a fuzzy logic control system with feedback over
networks to learn from video frames and detects fire based an Arduino microcontroller system for fire detection for
on fire signatures such as smoke and flames. Moreover, both homes and automobiles. )e automatic system con-
Bayesian networks are also employed for fire detection [10]. sisting of flame sensors, temperature sensors, smoke sensors,
Deep learning has progressed significantly; neural net- and a re-engineered mobile carbon dioxide air-conditioning
works can learn data with multiple levels of abstraction and unit was tested on amedium-sized physical car. Results show
automatically extract features from data, as evident in the that the automobile fire detection and control system devoid
paper published by Tao et al. in [11]. Motivated by the of false alarms detects and extinguishes the fire under 20
successes of deep learning in the area of computer vision and seconds. An innovative, very promising solution module for
image processing, a convolutional neural network (CNN) hardware implementation in fire detection and control for
was used to overcome the limitations of a sensor-based automobiles has been developed by using new algorithms
system. CNN was used to process video feeds from a sur- and fuzzy logic. )e data from the sensors are fussed to
veillance camera and alert in the event of possible fire de- enable accurate detection of fire based on fire signatures.
tection. To account for fire detection in wide areas, a camera )ere has been an increase in the number of publications
surveillance unit was integrated into the system’s architec- dealing with video processing to detect fire, particularly with
ture for ease of processing images of fires over wide areas of the usage of convolutional neural networks. Frizzi et al. [12]
the installed units. Most sensor-based systems are generally proposed a convolutional neural network for video fire and
limited in terms of their proximity to the fires they tend to smoke detection. )ey implemented a neural network with
detect, the closer they are to the fire, the better the detection nine layers. To detect fire and smoke in a video frame, a
results; thus vast areas can be quite daunting to detect fires sliding window of size 12×12 is applied to the last feature
using only sensed fire signatures like flame, smoke, and heat. map generated from the network. )e authors chose to use a
Consequently, sensors only are not useful in large areas since small network to increase the classification time. )e pro-
they require proximity to the signatures of the fire, as posed system was, however, designed to detect red-fire and
stipulated by Frizzi et al. in [12]. had some challenges with smoke detection.
In this paper, we present the design and development of Muhammad et al. [17, 18] proposed a cost-effective fire
a fuzzy logic-based multisensor fire detection system with detection CNN architecture for surveillance videos with less
both real-time web-based and mobile-based alert notifica- computational time and memory footprints. )eir model is
tion systems with the capability for image processing of fires inspired by GoogLeNet architecture, considering its rea-
using CNNs. We adopted the fuzzy logic approach to fire sonable computational complexity and suitability for the
detection because it simplifies the fusion of data from the intended problem compared with other computationally
Journal of Engineering 3
expensive networks such as AlexNet. To balance the effi- in Figure 1. )e hardware components of the sensor-based
ciency and accuracy, the model is fine-tuned considering the fire detection unit, as shown in Figure 2, is a device meant to
nature of the target problem and fire data. Experimental be installed in some premises similarly as a conventional
results on benchmark fire datasets reveal the effectiveness of smoke detector. At a high level, it comprises sensors and a
the proposed framework and validate its suitability for fire global system for mobile communication (GSM) module
detection in closed circuit television (CCTV) surveillance connected to a microprocessor that runs the fuzzy logic fire
systems compared with state-of-the-art methods. detection program.)emicroprocessor polls the sensors at a
In the paper byWang et al. in [19], they proposed a novel regular interval and runs the inputs through the fuzzy logic
algorithm based on a CNN and support vector machines application. If it concludes that a fire has been detected, a fire
(SVMs) for fire detection in infrared (IR) video surveillance. alert message is sent out through the web portal and the
To improve the performance of IR fire detection, they de- mobile app through the management information systems
velop a 9-layer convolutional neural network named IRCNN (MISs) to the occupants of the premises and the nearest fire
instead of traditional empirically handcrafted methods to station. If sending the message over the data connection is
extract IR image features. )ey utilized a linear support unsuccessful, then it sends the message out via short message
vector machine for training with extracted features to detect service (SMS). )e fire detection unit comprises the physical
fire. Experimental results show that their approach achieved components, including the sensors and the microprocessor
both high precision (98.82%) and high recall (98.58%) on board, and the software that embodies the fuzzy logic fire
infrared (IR) flame dataset and real-time detection. detection algorithm and essentially drives the system.
Different fire signatures such a flame, smoke, and heat )e software subsystem is that nonphysical part of the
were used for fire and smoke detection using CNN by dif- fire detection unit, which is concerned with reading inputs
ferent researchers [20–26]. Some authors extended their work from the sensors, determining whether the readings are
to include the detection of forest fires and enable fast response indicative of a fire or not, and raising alerts in cases of fires.
time for firefighting and the performance of rescue opera- )e sensors used, respectively, in [28–31] were chosen based
tions. Moreover, indoor fire detection using gas chemical on detection range, size, and cost. By adopting a multisensor
sensing provides faster fire alarm responses than conventional approach, the need to fuse the data arises. Fuzzy logic
smoke-based fire detectors. Gas-based fire detection could provides an easy way of dealing with uncertain data from
provide an additional level of safety to building occupants in multiple sensors by aggregating these to make a decision.
the event of fire [27]. But, these detectors rely heavily on one Fuzzy logic is a type of logic that tries to mimic the human
or two of the three fire signatures, which are inadequate for brain by incorporating the imprecision with which humans
effective fire detection and firefighting. make decisions. As a practical example, if a human is tasked
A real-time fire detection method combining AdaBoost, to describe the temperature of a cup of beverage, he or she
local binary pattern (LBP), and a convolutional neural would typically just say that it is hot, cold, cool, warm, not
network in video sequence was produced by Maksymiv et al. too hot, etc. rather than stating that it is 25°C or 52°C as
in [28]. )e proposed framework for emergency detection would a computer. Fuzzy logic is therefore based on im-
consisted of two main parts; the first part generates regions precision, and it uses linguistic variables, defined as fuzzy
or areas where flame or smoke may be present in the video sets, to codify common sense and hence approximate human
stream. )is operation is performed by the AdaBoost and reasoning. In this regard, each of the three fire signatures
LBP combined. )en a final classification is done by CNN. (variables) three states (sets) are defined and described by
)e generation of the possible regions reduces the com- membership functions. )e output is also defined as three
putational time of CNN significantly but is prone to a high possible states (sets): no fire, potential fire, and fire. Fuzzy
rate of false positives. rules describing combinations of the various sets which
)is research paper adopts the strengths of the real-time indicate a fire or no fire or a potential fire are coded into the
fire detection methods reviewed using convolutional neural system.)en as the system operates, it takes the raw readings
networks on video sequence frames and integrates that with from the sensors and fuzzifies and defuzzifies them to give an
proximity detection of fires using sensors that fuse all the output. )us, rather than saying there is x units of fire, it will
three fire signatures for early detection and alerts notifica- say there is a fire, or there is a potential fire, or there is no fire.
tion while providing navigational aid to the scene of fire for )e basic configuration of the fuzzy logic unit of the fire
the fire rescue team to respond appropriately.)is novelty in detection system is shown in Figure 3.
the choice of using both scalar and vision sensors overcomes )e size of the rule base of a fuzzy logic system is affected
the inherent problems highlighted in the review of fire by its inputs and sets used. A reduced number of fuzzy sets
detection systems. results in a reduced rule base and vice versa. For each sensor
input (linguistic variable), three fuzzy sets were created using
3. System Design and Development grade, reverse grade, and triangular membership functions.To determine these membership functions, data collected
3.1. System Overview from each sensor during tests was used to tune the mem-
bership of the various fuzzy sets. We used the center of area
3.1.1. Sensor-Based Fire Detection Unit. )e general over- (CoA) defuzzification method to obtain the system output,
view of the hardware module design and software imple- which gives the likelihood of a fire. Figure 3 shows a basic
mentation of the multisensor fire detection system is shown configuration of a typical fuzzy logic system. Figure 4 shows
4 Journal of Engineering
Web portal+ 
service (i) Management Information 
(i) Detects fire scenarios System (MIS)
(ii) Sends alerts to web portal. (i) Management Information 
(iii) SMS alerts. System (MIS) (ii) Receives alerts on fire detections
(ii) Sends alerts to mobile app.
Fire detector (iii) Sends alerts to fire service. Mobile app
Figure 1: General overview of the system.
Flame sensor
Smoke sensor GSM module
Beaglebone
board
Temperature sensor
Figure 2: Sensor-based detection unit of the system.
Fuzzy rule 
Input Fuzzifier Defuzzifier Output
Fuzzy 
inference 
Figure 3: Basic configuration of a fuzzy logic system.
Low Medium High Cold Normal Hot 1 Far Not far Near1 1
0 20 50 70 100 20 30 40 90 0 300 500 800
(a) (b) (c)
Figure 4: Fuzzy membership sets. (a) Smoke density. (b) Temperature. (c) Flame.
Journal of Engineering 5
the membership functions for each of these sets. )e am- Table 1: Rules for fire status when the smoke is low.
bient smoke density data from the smoke sensor were Flame intensity
grouped into three fuzzy sets (low, medium, and high). )e
temperature sensor readings have been classified into cold, Far Not far NearAmbient Cold No fire No fire Potential fire
normal, and hot fuzzy sets. )e fuzzy rules for fire status are temperature Normal No fire No fire Potential fire
shown in Tables 1–3 for different scenarios of smoke levels. Hot Potential fire Fire Fire
When the three sensor inputs are supplied to the system,
the inputs are then fuzzified using the various membership
sets. )e output of this process gives the fire status where a Table 2: Rules for fire status when smoke is medium.
fire alert may be activated.
Based on the output of the system, a fire event is either Flame intensity
dispatched to the web-based notification system, and house Far Not far Near
owner(s) or no action is taken. )e flow diagram for the fire Ambient Cold No fire Potential fire Fire
detection system is shown in Figure 5. temperature Normal Potential fire Fire Fire
Hot Potential fire Fire Fire
3.1.2. Convolutional Neural Networks. Convolutional neural
networks are multilayer artificial neural networks designed Table 3: Rules for fire status when smoke is high.
to handle two-dimensional input data like images [32]. It is
inspired by the mode of operation found in the visual Flame intensity
perception of living creatures [18]. Far Not far Near
Figure 6 shows the structure of CNN. )e layers of the Ambient Cold Potential fire Potential fire Fire
network are made up of multiple two-dimensional planes. temperature Normal Potential fire Potential fire FireHot Fire Fire Fire
Each 2-D planes consists of several neurons that make CNNs
suitable for handling image data.
)e presence of a convolutional layer and a pooling
layer in the architecture enables easy extraction of local
features and reduction of complexity as stipulated by Pei
in [33]. Register self on web app
)e Inception structure in GoogLeNet [34] can extract
more relevant features with less computational effort by
establishing and exploiting the sparse structure, thus im-
proving the accuracy of the network. GoogLeNet has 22 Read from sensors
layers, deeper than AlexNet’s 12 layers, but the parameters
are only one-twelfth of AlexNet’s, and the accuracy is higher. Read [Error = true]error?
)e basic structure of GoogLeNet is the Inception module, [Error = false]
which is shown in Figure 7. )e Inception module has two
Fuzzify and defuzzify
main features, namely, (1) reunion of convolution cores with
multiple sizes and (2) dimension conversion using 1× 1 [Output = potential fire] Fuzzy [Output = fire]
convolution [34]. Experiments show that, although di- output
mensionality reduction is first carried out, as long as the Alert users Alert fire service 
output dimension remains unchanged, the process of di- connected to place with SMS fallbackwith SMS fallback [Output = no fire]
mensionality reduction in the middle will not affect the
accuracy of the network. Inception V3 further decomposes a Alert users connected to place 
larger two-dimensional convolution into two smaller one- with SMS fallback
dimensional convolutions. )at means 1×N convolution Wait for 15
followed by N× 1 convolution can replace N seconds×N convolu-
tion. )e set n � 7 is used to handle feature maps of 17×17 Cache fire service and users 
image size [34]. phone numbers for SMS 
In the convolution operation, feature maps are generated fallback
by applying kernels of different sizes to the input data. A Figure 5: Flow diagram for fire detection software.
pooling operation is performed on the feature maps with
maximum activations from small neighbourhood in the
feature maps, as published by Muhammed et al. [18, 36]. )e efficient manner. For the intended classification task, the
pooling operation handles the computational overhead of Inception-V3 module was used. Inception-V3 is the 3rd
passing the features extracted directly to the classifier, as version in a series of Google’s deep learning convolutional
proposed by Aloysius and Geetha in [32]. )is approach architectures. It was trained using a dataset of 1,000 classes
using pooling operation is essential for images with high from the original ImageNet dataset, which was trained with
resolution and large sizes to be handled in a computationally over 1 million training images.
6 Journal of Engineering
Convolutional Fully-connected
layer 1 Convolutional layer
36 layer 226
12
11 6
3 3 2
7
3 3 2
11 612 7 3
36 26 9 3
3 Max pooling
9
9 layer 2Max pooling Output
3 layer 1 layers
Input layer
Figure 6: Architecture of CNN.
Filter
concatenation
3 × 3 convolutions 5 × 5 convolutions 1 × 1 convolutions
1 × 1 convolutions
1 × 1 convolutions 1 × 1 convolutions 3 × 3 max pooling
Previous layer
Figure 7: Inception module with dimension reduction. Source: adapted from [34, 35].
In traditional classification learning, to ensure the ac- instances, or features in massive data sets can be migrated to
curacy and reliability of the training classification model, micro data sets, thus improving the generalization ability
there are two fundamental assumptions, namely, (1) the and recognition accuracy of the model.
training samples for learning and the new test samples must With the transfer learning technique which allows an
satisfy the same distribution independently and (2) sufficient existing model to be retrained, resulting in a significant
training samples must be available. However, as time goes reduction in the training time, we retrained the Inception
on, previously available labelled sample data may become modules with over 1000 images of both instances fires and
unavailable. Also, labelled sample data are often scarce and nonfire situations.)e architecture of the Inception model is
difficult to obtain. shown in Figure 7.
Transfer learning is a new machine learning method that )e Inception module performs multiple convolutions
uses existing knowledge to solve problems in different but on given input and decides which one is best for the image.
related fields [37]. It relaxes two underlying assumptions in )is was necessary to handle the issue of varying resolutions
traditional machine learning, aimed at migrating existing of images and enable efficient classifier computations.
knowledge to solve learning problems in target areas where
only a small amount of labelled sample data are available.
)emore the factors shared by two different fields, the easier 3.1.3. Web Application. )eweb portal provides an interface
it is to transfer learning. Transfer learning can be divided for general users, manufacturers, and the fire service, while
into three main categories: firstly, case-based transfer the web service serves as a link between the fire detector and
learning; secondly, feature-based transfer learning in iso- the mobile app. It is the backing store for the entire system. It
morphic space; thirdly, transfer learning in heterogeneous performs the same functions as the mobile application and
space [37]. )erefore, through transfer learning, relevant informs firefighters nearby the detected fires and enables the
Journal of Engineering 7
manufacturers to track production units. )e use case di- CNN model is run on the data, and if a fire is detected, the
agram for the web application is shown in Figure 8. alerting system is thereby triggered.
3.1.4. Mobile Application. As our mobile phones are handy 4. System Implementation and Testing
devices, it is expedient to have a mobile app to complement
conventional siren. )is way, in the cases of a fire incident, )e convolutional neural network was trained on over 1000
users (occupants and owners of the place on fire) can be images of instances of fire and nonfire situations. )e data
notified wherever they are. After a person acquires and were collected from online image data sources. Sample
installs the fire detection unit in an area of his/her choice, he/ images used for the convolutional neural networks are
she can mark that place within the app by providing a name shown in Figure 13. )e model was trained through 800
andmarking its location on amap.)e user can then add the steps with a batch size of 100 and the 0.01 default initial
fire detection units installed to that place hence creating an learning rate. As a principle for the training of neural
association between the fire detection units. )en that user networks, the dataset was split into training and testing sets.
may add other users to that place by entering their e-mail )e module was trained on Google Colab, Google’s free
addresses. )e use case diagram for mobile application cloud service for Artificial Intelligence (AI) developers.
development is shown in Figure 9. Colab helps develop deep learning applications on a graphics
processing unit (GPU) for free. )e GPU specifications are
12GB GDDR5 VRAM having 2496 CUDA cores [38]. To get
3.2. SystemArchitecture. )e system architecture consists of an inference from the model, test video streams were used as
both hardware and software components. )e hardware input and passed through to the system. )e classifier
components comprise of the multisensor fire detection outputs probabilities for the two classes: “fire” and “no fire.”
system, which will have the installation of the software )e class with the maximum probability score is considered
components when the user purchases the unit and installs it as the result of the classifier. )e classifier module was
at residential, commercial, or industrial premises. )e user implemented with Google’s TensorFlow. TensorFlow is an
will configure the system based on the number of units open-source software library provided by Google for nu-
purchased. For enhanced fire detection, a surveillance merical computation using data flow graphs. [35, 39, 40].
camera unit is incorporated as part of the hardware system After the training the network on the data, a classifier with
implementation, which will continuously monitor the an accuracy of 94% was achieved. )e classification process
premises and send the video feed to a centralized server for is as follows; the video feed is preprocessed, and frames are
fire incident detection and alert notification. In the case of extracted from the video. )e extracted frames/images are
fire detection, the owner of the property will be alerted and classified to determine the condition of the area.
also an alert will be sent to the fire service for the fire and )e system hardware specification for the different
rescue crew with locational mapping and navigational in- sensors and system components are shown in Table 4.
structions using Google Map API. )e overall system ar-
chitecture is depicted in Figure 10. Based on the
development and testing of the multisensor fire detection, 4.1. System Testing and Results. Given the requirements of
the hardware components were configured according to the the various divisions of the system, they were tested to verify
diagram shown in Figure 11, and the actual hardware that they met those requirements. For the sensor-based fire
implementation for fire detection is also given in Figure 12. detection unit, tests were run in the lab using strong fumes
In order to enhance fire detection over a wider area that has a from the methylated spirit to trigger the smoke sensor, a
monitoring camera installation, we employ convolutional flame from the matchstick triggered the flame sensor and the
neural networks for efficient video processing during the temperature sensor. )e readings that were taken from the
testing phase of the development. )is architecture, with the sensors were logged to files and inspected to see if they
incorporated video processing capability, is depicted in reflected the presence and absence of the various fire sig-
Figures 10 and 12. natures. Logs were also kept for all requests made to the web
In the development of the video processing unit of the service so that we could ascertain whether the fire detection
fire detection system, we considered the processing speed of unit was able to send the alert when a fire was detected.
the entire system. )at is because, in the event of a fire Occasionally, we also disabled the Internet connection on
outbreak, an alert of the fire must be delivered on time for the fire detector to verify if it sent SMS alerts under such
necessary measures to be taken. Since the data stream being circumstances. Field tests were conducted in the lobby
fed to the convolutional neural network is primarily a video outside the lab and in the corridor at the School of Engi-
stream, there was the need for a high processing power to neering Sciences, University of Ghana, Legon, with fire from
run the recognition model on the data feed. Due to this need, burning paper. )e results of the mobile application de-
the option of running the recognition module on the velopment and test runs, as well as web application interface,
Beaglebone microprocessor unit was not an efficient option are shown in Figures 14(a)–14(e). )e Google API enabled
to consider due to its memory capacity limitation. )e navigation to the location of the detected fire. It gives fire
prospect of incorporating a dedicated server to run all video service personnel quick response time as it gives them the
processing was therefore employed. Hence, the video stream time of day navigation to avoid heavy traffic jams along the
from the surveillance camera is sent to the server where the path to the location.
8 Journal of Engineering
Fire service web portal
Add fire stations
View all fire stations
on map
Add fire hydrants
View all fire hydrants
on map
Receive alerts on fire detections
Fire service nearby
Get driving directions from fire
station to fire location
Get driving directions to the
nearest fire hydrant
Figure 8: Web application use case diagrams.
Mobile app
Manufacturers web portal
Add new place
Input details of detectors
View all places on map produced
Setup a new fire detector
User
View detections from
detectors Manufacturer View logs coming from
detectors
View logs from detectors
Figure 9: Mobile application use case diagrams.
)e field tests performed for the different components there is an incidence of a fire or otherwise; the classes for
and scenarios, and the corresponding results are given in classification are “fire” and “no fire.” Figures 15(a) and 15(b)
Table 5. show the results obtained from test instances of fire and no
)e video classification task for the system’s use case is a fire situations, respectively. )e test cases used considered
basic binary classification task.)e system indicates whether the occurrence of fire both indoors and outdoors from the
Journal of Engineering 9
Dedicated server
CNN detects fire
Send alert [SMS/push 
notification]
Surveillance camera To owner
Fire service
For each device Sensor unit detects
produced fire
Sensorbased detection
Deploy application onto unit
it
Add a place
View all places
Add another user
Add a detector
View detectors
Ship to stores Register user
Figure 10: System architecture.
MQ2 smoke
sensor
Beaglebone black
Antenna
GSM Module
GSM module
E603450 7B20
E7B02-D60-1 Infrared flame
1100mAh 3.7V sensor
DHT11 temperature
sensor
Battery
Fritzing
Figure 11: Sensor-based detection hardware unit.
surveillance camera. Table 6 shows the results of the test outputs from 0 to 4096. )e sensor units’ tests conducted
cases with corresponding accuracy values obtained. were intended to ensure the accuracy of each sensor. From
the result obtained from the temperature sensor, we ob-
served an error margin of ±2°C for each reading. For the
4.2. Results and Discussion. )e Beaglebone microprocessor smoke sensor, a higher value indicates high smoke obscu-
was central to the system since it performs all the processing ration and vice versa. With three output states of the fuzzy
functions. )e smoke and flame sensors, being analog de- logic algorithm, fire alerts only get activated when either the
vices, rely on the ADC on-board the microprocessor to potential fire state or fire state pertinence is high. Unlike a
perform the conversion. )e ADC on-board produces single signature-based detection system which operates by
+ –
10 Journal of Engineering
Training data set
Convolutional Fully-connected
layer 1 Convolutional layer
36 26 layer 212
11 63 3 2
7
3 311 212 7 6 3 Prediction
36 263 9
3
Max pooling
9 9Max pooling layer 2 Output
3 layer 1 layers
Input layer
Video feed Convolutional neural network
Testing data set
Figure 12: Video processing unit on a dedicated server.
(a) (b) (c) (d)
(e) (f ) (g) (h)
Figure 13: Sample images for classes.
comparing the sensor output to a set threshold, our mul- warning to be issued. Sample simulation results from the
tisensor approach decides based on the weight of each input. fuzzy logic algorithm are shown in Table 5. )e temperature
)is implies that the system can operate without necessarily values are given in degrees.
relying on all signatures. )is means the system can detect a )e video classifier performed very well on the tests run
flaming fire that produces flame and heat with little or no on the classifier module. To avoid the instances of false
smoke. )is property of the system allows for potential fire alarms being triggered, a threshold for the classifier
Journal of Engineering 11
Table 4: Hardware specification.
Component Specifications
Temperature sensor DHT11 temperature sensor (digital)
Flame sensor Infrared flame sensor (analog)
Smoke sensor MQ2 gas sensor (analog)
Microprocessor Beaglebone black rev. C
Power 5V 2A 5.5mm∗ 2.1mm barrel power adapter
GSM module SIMCOMM SIM800H-based Adafruit FONA 2G GSM breakout board
uFL antenna Sticker type 2 dBi uFL antenna
Li-poly battery 1200mAH 3.7V rechargeable battery
Perforated board 4 cm× 6 cm perforated board
Resistors 22 kΩ, 10 kΩ and 4.7 kΩ 1% resistors
Surveillance camera 5 megapixel camera with accessories
(a)
(b)
Figure 14: Continued.
12 Journal of Engineering
(c)
(d)
Figure 14: Continued.
Journal of Engineering 13
(e)
Figure 14: (a) Mobile application screenshots. (b) Web application home page. (c) Detector inventory. (d) Entity-relationship diagram.
(e) Web application map navigation.
Table 5: Field test results.
Test Expected result Remarks
)e fire detection unit boots automatically sets It was successful most of the time
Power on fire detection unit its date and time, tells the web service that it is Occasionally it failed to automatically update theonline, and starts running the fuzzy logic date and time;
detection algorithm a fix for that is soon to be tested
Leave the fire detection unit running )e fire detection unit keeps running the main )e system kept running;
under normal environmental detection program and keeps logs of readings one of the units, however, overheated; this was
conditions for three days from sensors attributed to the lack of a proper cooling systemon the microprocessor board
)e readings from the various sensors vary with For all the tests conducted, the system was able to
Expose the fire detection unit to fire the intensity of the flames, temperature, and detect the fire; beyond a 1-meter radius; however,smoke; the fuzzy logic algorithm classifies the the detected fire level had to be significant before
inputs as a definite fire and sends an alert the system could detect it
Expose the fire detection unit to very
high levels of only one or two fire )e fuzzy logic algorithm classifies the input as a
signatures potential fire
Successful
Send alert to web service on the )e fire detection unit makes a request to the
detection of fire web service to alert all concerned users and the Successfulfire service about the detected fire
Send alert via SMS on detection of )e fire detection unit sends the fire detection
fire and absence of Internet alert via SMS to all the concerned users and the Successful
connection fire service
Video output for classification tasks.
confidence was set. Hence, the alarm is only triggered into the convolutional weights that help speed up the
when the confidence is greater or equal to the threshold. model. Figure 16 shows a plot of the cross entropy against
)e aim is to detect a fire from the video stream with very the number of steps.
high accuracy and trigger an alert as quickly as possible. To )e cross entropy is a statistical measure of the error
boost the speed of the classifier, TensorFlow’s “opti- between computed outputs and the desired target outputs
mize_for_inference” script was used to remove all un- of the training data [41]. From the graph, the cross en-
necessary nodes in the module. )e script also does a few tropy reduces as the step size increases, an indication of a
other optimization processes like normalizing operations functional and suitable classification module. )e video
14 Journal of Engineering
(a) (b)
Figure 15: (a, b) Screenshots from the video classifier test.
Table 6: Video output results for the fire detection and classification.
Scenario Class detected Accuracy
Indoor (no fire) No fire 0.89
Indoor (fire) Fire 0.91
Outdoor (no fire) No fire 0.90
Outdoor (fire) Fire 0.99
Cross entropy
0.6
0.5
0.4
0.3
0.2
0.1
0
0 100 200 300 400 500 600 700 800 900
Steps
Figure 16: Graph of cross entropy against step size.
Table 7: Performance evaluation of subsystem modules.
Subsystem Performance remarks
Microcontroller unit 100% )e microcontroller works all the time
GSM module unit 80% GSM mobile is not always reliable due to network problems and power requirement issues
Sensor unit 80% Sensor data spikes sometimes lead to unreliable data from the sensors
Convolutional neural
network 94% )e CNN unit works as expected to provide high accuracy levels for detected fires from video feeds
Fuzzy logic system 90% Due to test setup constraints, the extensive fire test could not be conducted, hence the incorporation ofthe surveillance camera hardware system for broader area coverage
Fire notification
subsystem 100% )e system shows fire notifications in real-time
Table 8: Performance evaluation of fire detection.
Smoke Temperature Flame Status
647.1 83.31 0.00 Fire (0.708)
338.92 25.00 844.30 Fire (0.602)
548.5 28.93 918.49 Potential fire (0.459)
Journal of Engineering 15
classifier also supported different video resolutions since [3] S.-J. Chen, D. C. Hovde, K. A. Peterson, and A. W. Marshall,
the Inception module performs multiple convolutions on “Fire detection using smoke and gas sensors,” Fire Safety
input and decides which kernel size suits the image best. Journal, vol. 42, no. 8, pp. 507–515, 2007.
Based on the performance evaluation and testing, all the [4] Z. Tang, W. Shuai, and L. Jun, “Remote alarm monitor system
subsystem modules of the developed hardware and software based on GSM and ARM,” Procedia Engineering, vol. 15,
system were tested, and the results are recorded and given in pp. 65–69, 2011.
Tables 7 and 8. )e magnitude of the sensed parameters [5] R. C. Luo and K. L. Su, “Autonomous fire-detection system
(such as smoke, temperature, and flame) received from the using adaptive sensory fusion for intelligent security robot,”IEEE/ASME Transactions on Mechatronics, vol. 12, no. 3,
sensors used, enabling easy fire detection as they offer greater pp. 274–281, 2007.
flexibility in handling imprecision and nonlinearities. [6] K. L. Su, “Automatic fire detection system using adaptive
fusion algorithm for fire fighting robot,” in Proceedings of the
5. Conclusion and Recommendation 2006 IEEE International Conference on Systems, Man and
Cybernetics, pp. 966–971, Taipei, Taiwan, October 2006.
A multisensor fire detection system using fuzzy logic has [7] C. Cheng, F. Sun, and X. Zhou, “One fire detection method
been designed, developed, and deployed successfully. )e using neural networks,” Tsinghua Science and Technology,
fuzzy logic detection unit was augmented by a convolutional vol. 16, no. 1, pp. 31–35, 2011.
neural network to classify video stream to detect and alert [8] C.-y. Lian, “Design of intelligent fire alarm system based on
homeowners of the occurrence of a fire. Unlike the existing GSM network,” in Proceedings of 2011 International Con-
fire detectors, this system employs a multisensor approach ference on Electronics and Optoelectronics, vol. 1, no. 1,
whereby the outputs of three sensors sensing three different pp. 393–396, Dalian, China, July 2011.
fire signature parameters (smoke, flame, and temperature) [9] T. Ono, H. Ishii, K. Kawamura et al., “Application of neural
contribute to the fire alert decision and hence produce a network to analyses of CCD colour TV-camera image for the
more reliable fire detection system devoid of false alarms. detection of car fires in expressway tunnels,” Fire Safety
Experimental results obtained by introducing strong fumes Journal, vol. 41, no. 4, pp. 279–284, 2006.[10] C. Jing and F. Jingqi, “Fire alarm system based on multi-
from methylated spirit to trigger the smoke sensor, a flame sensor bayes network,” Procedia Engineering, vol. 29,
from the matchstick triggered the flame sensor, and the pp. 2551–2555, 2012.
temperature sensor to the device shows that the system can [11] C. Tao, J. Zhang, and P. Wang, “Smoke detection based on
effectively enhance the reliability of a fire detection system. deep convolutional neural networks,” in Proceedings of the
When a fire alert is detected, the system automatically re- 2016 International Conference On Industrial Informa-
ports them to the fire and rescue service and also notifies the tics—Computing Technology, Intelligent Technology, Industrial
owner(s) on their cell phones. )e system also provides the Information Integration (ICIICII), pp. 150–153, Wuhan,
fire and rescue service with a map-assisted navigation system China, December 2016.
to help locate the scene of a fire outbreak which is very [12] S. Frizzi, R. Kaabi, M. Bouchouicha, J.-M. Ginoux, E. Moreau,
helpful when there are poor house and street addressing and F. Fnaiech, “Convolutional neural network for video fire
system in a neighbourhood. )ere is, however, a need to and smoke detection,” in Proceedings of the IECON
design a good packaging for the fire detection unit. )e 2016—42nd Annual Conference of the IEEE Industrial Elec-
package must be able to accommodate the entire unit while tronics Society, pp. 877–882, Florence, Italy, October 2016.
allowing light, air, and smoke to reach the sensor. A possible [13] R. Sowah, K. O. Ampadu, A. Ofoli, K. Koumadi, G. A. Mills,
improvement to the developed system is the incorporation and J. Nortey, “Design and implementation of a fire detectionand control system for automobiles using fuzzy logic,” in
of neuro-fuzzy to substantially improve the detection rate Proceedings of the IEEE Industry Applications Society Annual
and more images to train the network for higher accuracy. Meeting, vol. 2, pp. 3–5, October 2016, Portland, OR, USA.
[14] R. Sowah, A. R. Ofoli, S. Krakani, and S. Fiawoo, “A web-
Data Availability based communication module design of a real-time multi-
sensor fire detection and notification system,” in Proceedings
)e data used for training the convolutional neural network of the 2014 IEEE Industry Application Society Annual Meeting,
are available online. pp. 1–6, Vancouver, Canada, October 2014.
[15] R. A. Sowah, A. R. Ofoli, S. N. Krakani, and S. Y. Fiawoo,
Conflicts of Interest “Hardware design and web-based communication modules of
a real-time multisensor fire detection and notification system
)e authors declare that they have no conflicts of interest. using fuzzy logic,” IEEE Transactions on Industry Applica-
tions, vol. 53, no. 1, pp. 559–566, 2017.
References [16] R. Sowah, A. R. Ofoli, S. Krakani, and S. Fiawoo, “Hardware
module design of a real-time multi-sensor fire detection and
[1] World fire statistics CTIF—international association of fire notification system using fuzzy logic,” in Proceedings of the
services for safer citizens through skilled firefighters, 2019, 2014 IEEE Industry Application Society Annual Meeting,
https://www.ctif.org/world-fire-statistics. pp. 4–9, Vancouver, Canada, October 2014.
[2] Fire outbreaks cost Ghana over GHC1 million in 2012 [17] K. Muhammad, S. Khan, M. Elhoseny, S. H. Ahmed, and
general news 2012 https://www.ghanaweb.com/GhanaHomePage/ S. W. Baik, “Efficient fire detection for uncertain surveillance
NewsArchive/Fire-outbreaks-cost-Ghana-over-GHC1-million-in- environment,” IEEE Transactions on Industrial Informatics,
2012-253460#. vol. 15, no. 5, pp. 3113–3122, 2019.
16 Journal of Engineering
[18] K. Muhammad, J. Ahmad, I. Mehmood, S. Rho, and [33] X. Pei, “Emphysema classification using convolutional neural
S. W. Baik, “Convolutional neural networks based fire de- networks,” in Intelligent Robotics and Applications, vol. 9244,
tection in surveillance videos,” IEEE Access, vol. 6, pp. 455–461, Springer, Berlin, Germany, 2015.
pp. 18174–18183, 2018. [34] C. Szegedy, W. Liu, Y. Jia et al., “Going deeper with con-
[19] K. Wang, Y. Zhang, J. Wang, Q. Zhang, B. Chen, and D. Liu, volutions,” 2014, http://arxiv.org/abs/1404.3978.
“Fire detection in infrared video surveillance based on con- [35] TensorFlow how to retrain inception’s final layer for new
volutional neural network and SVM,” in Proceedings of the categories tutorial—Youtube, 2018, https://www.youtube.
2018 IEEE 3rd International Conference on Signal and Image com/watch?v�m2D02eZTB4s.
Processing (ICSIP), pp. 162–167, Shenzhen, China, July 2018. [36] K. Muhammad, J. Ahmad, Z. Lv, P. Bellavista, P. Yang, and
[20] S. Khan, K. Muhammad, S. Mumtaz, S. W. Baik, and S. W. Baik, “Efficient deep CNN-based fire detection and
V. H. C. de Albuquerque, “Energy-efficient deep CNN for localization in video surveillance applications,” IEEE Trans-
smoke detection in foggy IoT environment,” IEEE Internet of actions on Systems, Man, and Cybernetics: Systems, vol. 49,
:ings Journal, vol. 6, no. 6, pp. 9237–9245, 2019. no. 7, pp. 1419–1434, 2019.
[21] C. Hu, P. Tang, W. Jin, Z. He, and W. Li, “Real-time fire [37] Y. Li, A. Wu, N. Dong, J. Han, and Z. Lu, “Smoke recognition
detection based on deep convolutional long-recurrent net- based on deep transfer learning and lightweight network,” in
works and optical flow method,” in Proceedings of the 2018 Proceedings of the 2019 Chinese Control Conference (CCC),
37th Chinese Control Conference (CCC), pp. 9061–9066, pp. 8617–8621, Guangzhou, China, July 2019.
Wuhan, China, July 2018. [38] Welcome to colaboratory—colaboratory, 2019, https://colab.
[22] J. Zeng, Z. Lin, C. Qi, X. Zhao, and F. Wang, “An improved research.google.com/notebooks/welcome.ipynb.
object detection method based on deep convolution neural [39] R. Phadnis, J. Mishra, and S. Bendale, “Objects talk—object
network for smoke detection,” in Proceedings of the 2018 detection and pattern tracking using tensor flow,” in Pro-ceedings of the 2018 Second International Conference on In-
International Conference on Machine Learning and Cyber- ventive Communication and Computational Technologies
netics (ICMLC), pp. 184–189, Chengdu, China, July 2018. (ICICCT), pp. 1216–1219, Coimbatore, India, April 2018.
[23] A. Filonenko, L. Kurnianggoro, and K. H. Jo, “Comparative [40] D. Demirovic, E. Skejic, and A. Serifovic-Trbalic, “Perfor-
study of modern convolutional neural networks for smoke mance of some image processing algorithms in tensorflow,” in
detection on image data,” in Proceedings of the 2017 10th Proceedings of the 2018 25th International Conference On
International Conference on Human System Interactions, HSI, Systems, Signals and Image Processing (IWSSIP), pp. 1–4,
Ulsan, South Korea, July 2017. Maribor, Slovenia, June 2018.
[24] Z. Yin, B. Wan, F. Yuan, X. Xia, and J. Shi, “A deep nor- [41] G. Nasr, E. Badr, and C. Joun, “Cross-entropy error function
malization and convolutional neural network for image in neural networks: forecasting gasoline demand,” in Pro-
smoke detection,” IEEE Access, vol. 5, pp. 18429–18438, 2017. ceedings of the International FLAIRS Conference, pp. 381–384,
[25] V. Sedlák, V. Stopjaková, and J. Brenkuš, “A real-time method Sarasota, FL, USA, August 2014.
for smoke detection in monitored forest areas,” in Proceedings
of the International Conference On Applied Electronics, Pilsen,
Czech Republic, September 2017.
[26] Z. Zhong, M. Wang, Y. Shi, and W. Gao, “A convolutional
neural network-based flame detection method in video se-
quence,” Signal, Image Video Process, vol. 12, no. 8,
pp. 1619–1627, 2018.
[27] J. Fonollosa, A. Solórzano, and S. Marco, “Chemical sensor
systems and associated algorithms for fire detection: a review,”
Sensors (Basel), vol. 18, no. 2, p. 553, 2018.
[28] O. Maksymiv, T. Rak, and D. Peleshko, “Real-time fire de-
tection method combining AdaBoost, LBP and convolutional
neural network in video sequence,” in Proceedings of the 2017
14th International Conference the Experience of Designing and
Application of CAD Systems in Microelectronics (CADSM),
pp. 351–353, Lviv, Ukraine, February 2017.
[29] DHT11 temperature and humidity sensor sensor—temp/
humid robot R Us, 2019, https://www.robot-r-us.com/vmchk/
sensor-temp/humid/dht11-temperature-and-humidity-
sensor.html.
[30] In-depth: HowMQ2 gas/smoke sensor works? and interface it
with Arduino, 2019, https://lastminuteengineers.com/mq2-
gas-senser-arduino-tutorial/.
[31] IR infrared 4 wire flame detection sensor module IR flame
sensor module detector smartsense for Arduino-in sensors
from electronic components, 2019, https://www.aliexpress.
com/item/32849642745.html.
[32] N. Aloysius and M. Geetha, “A review on deep convolutional
neural networks,” in Proceedings of the 2017 International
Conference on Communications, Signal Processing, and Sys-
tems (ICCSP 2017), pp. 588–592, Harbin, China, July 2017.