[0:00] Mr. Jotham Kildea ICONICS Solution Sales Supervisor
All right, thanks for joining for session today. We're going to be talking about “Rethinking Solutions for Your Connected Workforce”. For those of you online, there are two sessions. This is one. So glad you could join us. Today, this session has two presenters. I’m Jonathan Kildea, a Solution Sales Engineering Supervisor with ICONICS. I head up the team in the US. I've also got a colleague, Josh Obal who will be helping out with some of the demos and some of the technical aspects. He's a Senior Software Engineer with ICONICS. We're happy to be here. Just as a quick note for folks, on the Attendee Hub, if you're joining virtually, please do enter questions. I do want to have time at the end of the session for questions if you have them. Megan's going to help out reviewing them. And we'll look at those at the end. Feel free to type those in as you as you think of them as we go along.
[0:57]
So, what we’re going to cover today, I’ll give you a quick overview of what we kind of see is the current situation, the current environment in this space of how you can connect and facilitate remote work. This is going to get into some of the things that we do particularly around notification dispatch, which is one of the technologies we use for sending messages and notifications out to the field and keeping those people connected. Also looking at some of the mobility solutions that are available. What do you want to think about when you're designing and deploying a mobile solution to a remote workforce, got a couple of great demos set up with some real applications? We’ve got some stuff that we've put together with the engineering team that will go through, along with that last speaker, Shane Stevens, who's at Impact Automation. He's going to be talking about ICONICS application at Lake Cities Municipality Utility Authority. I know Ted mentioned a little bit about it earlier this morning. So, we'll see that in a bit more detail.
[1:53]
And as I mentioned, questions, definitely let us know what you think. So, they said a bit of the overview of what the situation as we see it; this is not authoritative. This is just some kind of compiled thoughts that I've put together over the years of working with these types of applications. So, here are some things that we have to watch out for. A big one is folks feeling cut off or disconnected from the application when they're not in the office. This is definitely a very common thing where somebody says, “Oh, something's gone wrong. I'm aware of something's being wrong. I need to get to the office to figure out what's going on.” That's kind of a possible problem because you're limiting yourself to only being accessed to the system when you're in the office, which is, of course, not always the case. This also causes a lot more back and forth where somebody is out in the field or in a factory. They're out to the site, so they have to go back to a system to check what’s going on then go back to the site. It’s that kind of repeated back and forth in order to diagnose a problem. Ideally, you want to minimize trips. You want to be able to have something that you can take care of on the fly.
[2:58]
I'd say that this one above all of the other topics is something that is already being shoved in everyone's faces, especially with COVID. Previously they kind of put up with the fact that they had to have this dual mode of being remote and in person. With COVID, a lot of situations that in-person is no longer an option. People have to be functional and active remote. And it's really driven that need for change and for having a solution for that.
[3:24]
Other things that we see: there is still today a lot of what we call islands of automation. This is where you’ve got a lot of different applications and systems that go into a solution like SCADA. We know that very well. CMMS systems for work order management, work order management systems, product management systems, all these different things that definitely have a purpose and a role, but frequently are not connected. They're islands; they're disconnected. They're not talking to one another. Any sort of transfer of data from these systems is, by and large, a fairly manual process that somebody has to transact. That's what we mean when we talk about an island of automation. This is something that, of course, we want to work to solve and improve.
[4:09]
How do you automate the flow between these systems? It still continues to be a problem today. So definitely we're going to talk about a bit today. And then also things about security. Right? I don't need to belabor the point. There are a lot of things going on right now as far as cybersecurity, particularly in SCADA systems, particularly in these remote worker situations. Just generally, the questions that we see is, “Where do we start?” The applications we interact with, traditionally had been in an air gap solution that they just said, “Hey, we're shut off from the world. We've got our own little network; we don't talk to anything.” Increasingly, that's not really a viable solution. And it doesn't take advantage of any of the new technologies out there today. So, a lot of the people that came from the model of “I can just shut myself off and I'm safe”, they have to reevaluate and say, “Well okay; maybe that's not going to work? Where do I start?”
[5:03]
Also needing to do things about risk factors. Once you've decided to start, you have this concern of, “Well, where are my threats? What's the thing I need to secure the most? What do I need to be aware of?” It's kind of a rapid learning process to try to figure that out. Another thing that we see is that people come into it thinking that cybersecurity is something that they can purchase, or implement, or just deploy some piece of software, and they're safe. A lot of things that have to do with cybersecurity are training and procedures as much as they are software and hardware solutions. So how do you teach people to be safe? How do you implement good practices that make sure that what you deploy is secure? And then the always the big question of “What are you overlooking? The biggest risk is the thing you hadn't thought of before. So, these are the types of things that people are very conscious of, as far as I see in the space, and something we want to talk a bit about today. So, with that, I'm going to jump into a little bit more in detail on what I was talking about as far as notification dispatch.
[6:19]
So, notification dispatch. I'm going to give you a high-level summary of what I think the goal you might want to achieve with this is, and then we're going to unpack that a bit more and look at “Well okay, how do you actually build that? What do you put together? What are the pieces you want to think about?” So, when we talk about notification dispatch, very simply, this is connecting to an alarm infrastructure. What do you have today that is an alarm or an event? Is that something you need to notify people about? How do you connect to that? How do you grab that information? Bringing in other systems, all those third party islands things? How do you bridge that gap and bring them in? What can you do to integrate all those different systems so that the actions you take are going to be globally applicable across your organization? And then what can you do? So, making smart choices about who you notify and how and when talking about tracking that effort application? So, when people are being notified, you have an audit log of that. What can you do to facilitate people being more effective once they have that information? These are all the things that we're trying to achieve when we talk about an automated dispatch system. There's a lot that can go into the interconnections between these different systems.
[7:34]
So, I want as much as possible to try to make it simple and approachable for you guys. So, an event; who knows what that is? That could be something broke; an alarm was tripped; there's a trespass alarm; a door was left open; something's running too hot, too cold, whatever - something happened. Let's just say you want to tell people that that happened. Take a workflow. This is where you have decision logic that says, “Okay, something's happened. I need to tell people, ‘” Who do I tell? How do I tell them? Where do I find them? Where do I find the information to notify them of?” That's the workflow. That's the logic that builds that. Then you've got the notification. How do you get that to them? What's the last leg of the journey? You've decided who to contact? And how, what actually makes that happen? Is that the question? And then of course, the recipient: What do they see? How do they respond? This is a hybrid of technology, as well as behavior and practices that we talked about.
[8:31]
So, I think for most applications, the part that's the least obvious of what the heck that is, is going to be the dispatch workflow. So, to give you a little bit more detail on that. When we talk about this, this is in ICONICS lingo, a workflow is something that you see like in the screenshot here where there's some action that's triggered it, and you can have whatever you want to happen from there. So, this is, in this case, a simple scenario of notifying somebody. You first look up how to contact them, how to find their phone numbers. And how do you get that information and then link it up to decisions about how you filter out who's the most appropriate person to notify first and how all that stuff is encapsulated in the workflow? This is kind of the core thought process that goes into our deployment of notifications to work this out in the field. So that's what we talked about when we talked about the dispatch workflow.
[9:35]
The other thing I wanted to look into a little bit more detail is the events. I was very brief on what an event is earlier, but an event in the ICONICS terminology would most commonly be something like an alarm from Alarm Work Server, from Hyper Alarm. It could be a fault from our fault detection diagnostics engine, or it could be a trigger, could be something from our data protocols. So BACnet and SNMP and OPC UA all have a standard for what is an alarm versus a data stream. So, any of those things could be a justifiable reason to trigger an event. That's what we talked about. We talked about event sources. Okay, so put that into the picture. Next, I want to talk a bit about the notification, how does that go out? In reality, there are several different technologies. These will sound pretty familiar to you when we talk about them. When we think about this, I would say where the industry is really leading towards is VOIP based technologies. These are things that you can access over an internet-based protocol. And, increasingly, these are being provided as some sort of SAS service from some company that says, “Hey, send us a message’ Send us who you want to send it to. And we'll take care of the rest.” We particularly like those because we understand cellular technologies. As a consumer, I don't know much about cellular technologies as far as sending people text messages when it's actually being sent over the wire. So, we use those services. They really know that industry very well making it very easy to abstract that complexity and to silo it from something you have to worry about. Definitely something we've broadly adopted in a lot of our applications. In a little bit more detail. There are several out there These I would say find like the best-in-class options. You can see the options here. We talk about simple things like email, a standard thing, you’ve been doing that for decades. Now when something happens, you send somebody an email. A lot of different services can do that. We can absolutely do it directly to your existing email server. You might also consider some of these services like Twilio, Avantage, or SendGrid because they have a lot more dedicated bandwidth for deploying emails in scale at high availability, reliance, and robustness that you can definitely take advantage of if that's a concern. SMS is another classic example; you'll see this in the demo. That just means, “Hey, something's gone wrong,” Send somebody a text message to let them know and give them a short little snippet of detail about what's gone wrong and maybe give them an opportunity to respond to that. That's going to be an SMS message again. VOIP technology is by and large these days. Same thing goes for voice. So, this is a text to speech. Somebody gets a phone call now that they answer. They get some pre-generated or dynamically generated text to speak smashes, and it explains the problem. Again, they have an opportunity to respond and react and address it. But just a different format really depends on the use case, on how people prefer to decide between those. Which one's best for them? Do you want to see something? Then it's a text message. It's a quick little glance. You can see something, but it can easily be forgotten. A phone call? You have to pick it up the phone and actually listen to something that's being addressed to you. So, a little bit more of a break from your schedule, but that's sometimes what you want. You want to have get people's attention. So, it depends on the right approach for the solution. And then WhatsApp. It's actually a category of a few different technologies. But What's App is the most common. For an idea of the traditional kind of hardware-based systems that you've merged to via VOIP. WhatsApp has always been an IP based solution. Again, sending a cryptic message to somebody really depends on what's the preferred method of notification within an organization. Some people might standardize on one thing, others might sound another. I'd say from our perspective, we don't want to force anybody into any one particular one. That's why we make these multiple options available. We looked at the market and said, “Hey, these are the best solutions that are out there. Let's make sure we can facilitate sending messages through these.” Some folks may prefer one or the other; it’s totally up to them.
[13:58]
So, taking this a little bit further: We said, “Okay, and when that happens, decide how to send it; decide which format gets sent. What does the worker see?” SMS messages look just like any other message. A phone call could be any other message. We can also leverage the MobileHMI which is an ICONICS app on the smartphone or tablet as a possible destination for that experience. Receiving messages to the MobileHMI app, also MobileHMI very importantly becomes a good platform for that bidirectional sharing of information. A worker that sees something on MobileHMI can see, of course, the alarms, all of the details and browse to other things. They can go beyond just seeing what was posted in an initial notification. So, this is definitely something we'd like to use on more robust implementations.
[14:47]
On that interface, what you see here is an example of how you might use this for notification dispatch. This is just a standard graphic within MobileHMI so there's really nothing crazy going on there? We talked all this morning about how to build and deploy graphics. That's an example of what you'd see there. There's also some fun stuff that you can use that gets telemetry data from the device. So, if you go down this path of this, there's a way that you can take information about the worker, where they are, where they're, what their availability is, for signal strength, or whatever, and incorporate that into how you notify somebody, we'll get into that a little bit. But it's a cool, cool implementation of how you might use this a little bit further. So, talk to all about that.
[15:31]
Also, acknowledgement. So, if you're familiar with the concept of alarm acknowledgement, this just means that this is a way for the operator to respond back. It's no longer an outgoing message, this can be a bidirectional conversation, where the user can acknowledge what happened on an alarm. And that's facilitated through most of the technology we talked about, I mentioned SMS, voice, WhatsApp, all these things can receive a message and then receive a response from the user. So, it can be just a quick control of that system from the interface that they're using. As I mentioned, that's facilitated both via the MobileHMI app, as well as through a lot of those direct protocols.
[16:18]
So still building, still adding some complexity here. This, of course, at any point we're talking about that that could be the solution that gets deployed. But I want to show you kind of give you some ideas of where this could go beyond that, as you grow and expand the complexity of it. Don't feel that all of these modules have to be deployed day one. I definitely recommend you kind of build to this approach over time. But it’s to get you thinking about some other things you could do in the workflow. There are other things, other fun things you can do to take advantage of information you might know about your workers, what their expertise is, or what their skills are, what their schedule is, what's who's available on off shift at different times, where they are kind of alluded to that with the app, all of these things in general, are you going to be coming from some other system. This is where we get into that “What can I do with my ERP system?” “What can I do with my CMMS system?” One of the things you can do is plug it into your dispatch to make more informed decisions about who do you contact. Don't contact the person that's on vacation. Don't contact the person that's off shift and not on call right now. But do contact the person that is on shift and happens to be a mile away from where the incidents happen. That's kind of the concept of what we're trying to do with all this information. All of these different things can be fed into the workflow. And at the end of the day you're trying to prioritize so that ideally you send the message to the person, that's going to be the best to respond to it first, and most efficiently. Of course, you always have a backup plan of who else to contact if and else and whenever something goes wrong. But hopefully, if your workflow is well tuned, you can make sure you contact the right person at the right time immediately.
[18:03]
So, I mentioned that a lot of this comes from third party systems. You absolutely can configure pretty much all of what we talked about there for schedules, availability, location, all of that within a standard solution. But I would typically recommend for larger applications, especially you look to “Do I already have this in a system somewhere else?” When I talk to a lot of applications that when we get into this discussion, yeah, I want to get these people notified, I want to make sure that they know what's going on, we find out well, they also have responsibilities while they're remote to interface to a work order system, or to interface to their corporate network and using Active Directory. So, we have this information about them in some capacity of what their availability is, where they are, what's going on.
[18:51]
My suggestion: don't reinvent the wheel. If you've got that configured, you've got that implemented, best just integrate it directly into the system. That's one of those connecting those islands’ opportunity to take what would be a normally a disconnected system and just jive it all together. So put that there, that's options over on the left of bringing all those different pieces together; that's really your best third-party integration opportunity.
[19:18]
Other things that you can do: This is not really impacting the ordinary flow of the system, but more of a design and reworking and making your system more efficient over time is the analytics that comes out of this. You see this in a few different ways. We've got this tool AnalytiX-BI, we really love using it now because it's really good for taking this just raw data and making some insights out of it. So, we take a lot of the system information of who was notified, how they're responding, how frequently do they respond, yes versus no to alarms. All of that information can be fed into tracking and analytics so that not only do you have a good audit log, but you also have the insights to say, “Alright, are there ways that I can more finely tuned my system in the future so that I'm doing a better job of notifying people and making sure that they're informed and can do their job?” So, again, the thing I want to stress here is that there is a lot of stuff you could do. Don't feel that that's all going to be a day one task. It's definitely going to be starting with that center column. Do I have a message there? Do I have an event? Do I decide what to do with it? Can I send it out to a user and can that user see it and hopefully respond to it? That's the core capability. You need to achieve everything else beyond that just iterative improvement optimizations and efficiencies.
[20:42]
A little bit more on the mobile experience. So, everything we talked about thus far is pretty much getting information out to the worker. I wanted to spend a bit of time talking about the workers out there, they see something come in, so what do they do about it? When I look at the value proposition of why you have a connected remote worker, one of the big benefits is going to be extending the reach of the existing system. If you've already got a SCADA system that is being used, it's being configured in operations. That's great for the person in the control room. Without a remote connected worker, you're totally blind once you leave the control room. So, taking that information that would be valuable in a control room, bringing it to anybody that's out in the field, that's obviously a simple example. Also, for that real time feedback, there's a lot of value and insight that somebody out in the field can provide over to someone in the control room in certain situations. You want to make sure that that information is fed back into the system, so it's not just captured out in the field and never left there. You actually incorporate this into your full dataset. Capture wherever they are.
[22:07]
And then something interesting to do is curating the experience. So, if you're a remote worker, it's not necessarily appropriate to see the same things at all locations. There might be situations that you want to curate a worker experience. Okay, you're at a site, it knows you are at the site, so let's bring up the site appropriate information. I shouldn't have to spend a lot of time figuring that out. I always look at the analogy of oil and gas applications. There are a lot of pump jacks, a lot of wells out in the field. It could just be a tedious process to scroll through your list of pumps and find the right one that you're at. And if you click on it, you hope you didn't pick the wrong one because you're getting wrong information. So, if there's some way you could say, “Hey, you pulled up to the site; it knows you're within 20 meters of that well, here's the dashboard for that site.” That's a lot of value that saves people time; that avoids mistakes. They're not looking at the wrong data, so there are a lot of benefits that can be gained there.
[23:03]
And then also the connectivity with other workers. Again, something that comes up a lot since COVID is how you make sure people are not totally cut off, that they still know what's going on. They don't see their coworkers face to face as often anymore, but they don't miss out on information. They don't miss out on Intel insights that are going between people. So, I use this example: it’s a clear example of how sometimes there's enormous value to being on site versus being in the control room. It's anathema to what we do because we do a lot of SCADA applications that are about building those dashboards. But there are just some things that being on site is better, right? Diagnosing a situation: if something's going wrong out in the field, you don't know. In the control room, you might see a SCADA system that just says, “Hey, here are the stats. There's something kind of wrong, but not sure what.” If you're on site, it might be blatantly obvious what's going wrong. You don't need a SCADA system to walk on site and say, “Oh, yeah, that's not supposed to do that.” So, there's value to the fact that somebody that's a remote worker that's on site can immediately capture that what's going on right now. They can also hopefully correlate it to what was happening. So, you see as a problem now, but how long has that been a problem? If I can pull up my system, and quickly, at a glance, say, “Oh, this happened last night at 9pm. And ever since then something has been wrong.” That's going to give you more context to what you need to do to fix what's going on in the system. Also, getting that information back, as I said, we want to make sure that the remote workers are feeding data into the system just as much as they're receiving data from the system. So, they can keep the team informed. If they see something like that which should be something that can be easily captured into the system quickly, so that information is not lost, and they know what's going on.
[25:04]
So, a big part of that is going to be acknowledgement. I'm going to talk about this in a bit more detail. The reason I use the term acknowledgement is that it comes from the OPC alarm events specification, is that it is a very standard feature of what an alarm event can handle. It's common across a lot of different things like BACnet and SNMP as well. That an acknowledgment just simply means that some user has chosen to acknowledge that something has happened. What that means to acknowledge something will depend from company to company and system to system. This is again an example of where process and how people handle it can be as important as the actual technical specification because what acknowledgement might mean to one company might mean dismiss this alarm. For another one, it might be acknowledgement which means that I am taking personal responsibility. I Jotham have seen this and have said, “This is my responsibility from here on out.” There are very different ways of handling the same type of alarm. Again, it depends on what is the corporate procedure, what is the practice and policy of how you handle acknowledgement. I'm tired of saying acknowledgement, so we also call ACH. So, an ACH is just shorthand for acknowledgement. Like I said, it could be anything. It could be, “I'm taking ownership of this.” It could be, “Sure, I’ll dismiss it.” It might be more of a multi-step alarm management, alarm management versus work order management. These have a lot of commonalities and similarities, but there are distinctions as far as how people handle those. So, ACH might mean something different to somebody depending on the different app or application. ACH is a common one because it's using the standard. But there are a lot of other ways to do this.
[26:49]
In general, just to point out that this is one of the quickest and easiest tools to enable something that's remote as a way of just taking action and decision making on what goes on an alarm. Again, depending on the process, it might be appropriate for somebody to look at something and say, “Yes, that's a problem. But I'm not taking responsibility for it. So, I'm not going to ACH it. Maybe I'll send it to the next user.” That's certainly a valid approach. Or maybe they do want to take the approach of acknowledgement, and say, “I'm going to take ownership of this.” It really depends on other things to look at, along with that is saying it ACH happens or say just an event happens.”
[27:27]
You should consider what else you want to integrate into the system to keep everything apprised. You don't want to have discrepancies in your data that one system is ahead of the curve knows a lot while others don't know a lot and out of sync. So, what else can we integrate? Can we do CMMS systems? Can we do integration to other people? These are all things that are going to have a lot of benefit to a remote worker over and above something that would be in operations in a controlled environment. In particular things on maps and notifications. Some of the interesting things we've been looking at and projects I wanted to highlight were geofencing or geolocation. So, this is kind of an amalgamation of different ideas. One is tracking where people and where assets are. So, we have good identity of the person that is here. This might be based on truck-based information. I know this fleet vehicle as at this location; a geofence is defining a region around something and saying, “I want to be notified when somebody enters or leaves that. That is a simple example. I want to be notified when somebody is there because they shouldn't be there. It's a controlled environment, and I need to make sure that I know who's at a site at any given time.” A positive reason to use a geofence would be to say, “Hey, this goes back to the oil and gas analogy, there's a pump out in the field. It's got some low priority, but things need to be fixed. But it's not worth sending somebody 20 miles one way to go out and have a look at it; it can wait.” But if somebody's driving by, it might be worth the swing by and have a look. So, a geofence is a great opportunity to say, “Let me define a scope of how far an alarm notification can reach? How important is this? Should there be somebody getting sent this regardless of where they are?” Or should I maybe scale it down to say, “Yeah, I want you to be informed of it if you're within a mile, but if not, it can wait. Or it'll be on the queue. But it's not going to be a high priority.” Those are the types of optimizations you can get to with some things like this and plays directly into remote workers because by and large, this is based on people that are mobile who are perhaps in transit. They're not always going to be at a fixed known location.
[29:45]
Another thing I want to touch on is what is the right approach, and there's not one answer. What is the right approach to enabling those remote workers? Is it one app that does everything? Or are there multiple apps that each do their respective things? There's pros and cons to each. And I'll be honest, it's like I said, it's not an easy answer to solve. One app does give you the advantage of less context switching. We'll see this in the demo that there are things where if I can jump from one thing to the next, it's the exact same UI, the exact same format, there's no load time, as a worker, that's nice. That's really seamless. I have all of my data in one spot. It is admittedly more to design. If you're designing one app that can do everything. Well. It has to do everything has a lot more integration to do, and it also might be a bit more work as far as linking all those things. As I said, there are a lot of islands out there, so how do you connect them all together and is it really efficiently? Not everything exposes all of its data. The flip side, multiple apps in here I'm talking about, “Okay, I'm out there. I've got my alarm acknowledgment app. I've got my work order app. I've got my Team's app. I've got my remote expert app. I've got all these different things going on, all of these tools that I can use.” But individual apps, as I said, are pretty easy to get going. Everybody out there has an app, ICONICS included. But it's going to be optimized towards its respective use. But just the opposite of the single app, it's not going to have a tight integration. There's going to be some things that you might provide more context: switching for a user say, “Oh, I figured something out. Okay, hold on. I have to keep that in mind because I have to jump over this other app, find somebody in contact them.” You want to find a sweet spot where to share the information you need to share, but not make the users waste time just rummaging through data to find what they have to do next. Right. So, I don't have a prescriptive answer for you. Just something to think about. If you're deploying a project, to keep that in mind of what do you want the user experience to be like and does that fit with what you think is efficient for your users?
[31:59]
Security. So, the security comes in in a few different places. When we talk about this remote connected as far as ICONICS is concerned. I just jotted down a few notes of things I wanted to convey that are important to think about. So, when you're looking at the mobile user interfaces, so mobile HMI, web HMI, these are technologies from ICONICS that deploy to the remote devices. Best practices. There's a lot of things out there, definitely make sure to use HTTPS encryption for the messaging. Not all that hard to set up. There are definitely tutorials on it and how to do it. It's a good idea. You want to make sure you're using an encrypted pathway for anything you're sending out online for sure. If you're familiar with the encryption, we can use self-signed, or CA signed or certificates authority signed certificates. If you really know what you're doing, self-signed are OK but by default, I'd recommend CA signed for standard applications. And also look at using DMZs. So, when you're deploying an iconic system, pretty straightforward to modularize it and make a dedicated web server versus your back-end server. That web server might appropriately be in a DMZ in a lot of situations. And that totally fits well with our architecture of what we typically deploy; something to think about. Let's see other things on third parties. So, as I mentioned, integrating to other things, you're not always going to be within the ICONICS scope. So how do you make sure that you're secure on that communication? So, REST and OPC UA, I think would be the kind of the most common interfaces that you'd use for third party here. I'm thinking of things like a work order system and ERP system. These are the more common protocols you might use. Both of them have good options for HTTPS and SSL encryption on that messaging, definitely use that; make use of that. That's a good idea.
[34:09]
And yeah, definitely watch out for things that use basic authentication. That's where general rule of thumb, if you look at the message that's being sent for a URL you're connecting to, if your password or your login account is in plain text in there, that's an old technology, don't use that please. Definitely use some things that do full and end encryption on that. And then also think this is not just a technology but think about what you are exposing. There are a lot of things that you can throttle in and out like what sort of data might be exposed via these API's. Not saying make it difficult for that third party to get data. But don't expose more than you have to; it's just good practice that you're exposing the things that are appropriate for that level of communication. And also think about things that are coming from third parties. So, I was talking about different integrations as a lot of other systems that we bring into a remote worker environment; make sure that those are encrypted as well. Same thing. Also, if you're sending data out, because we're talking about worker location, we're talking about worker information, their emails, their phone numbers, their availability, definitely think about GDPR requirements and constraints. I know that's technically a European constraint, but it's been pretty broadly adopted in the US just by default anyway. There are considerations for it within our applications to make sure that the deployment can be GDPR compliant. Make sure you think about it before you have any exposure of data that's not sending out private information.
[35:46]
So, what would you measure as success with deploying mobile application? The kinds of a things to think about if you're deploying a mobile remote worker application, ideally don't interfere with the normal work, right? If somebody is, by and large, in the industry, if somebody is out in the field, they're out in the field because they need to go do work that probably involves their hands. If it was something that could be done electronically, they could do it remotely. So, if you're out in the field, you’re probably out there to do some work. You don't want to have a technology that requires them to be constantly switching between “Okay, hold the tools and equipment. I have to put these down, go over here and look at my mobile device and go back.” Something really needs to be adaptive to them and also that is not bothersome. When we talk about notifications. Step one is “Cool, I can send a notification?” Step two is “Okay, I really only want some notifications. You don't want to be told everything”. So how do you dial in? What's the appropriate amount of information, so I'm not just distracting folks all day long with what's going wrong? Also, things that can be accessed quickly, we talked a little bit about the geolocation awareness. Can I send information appropriate to somebody in a correct location? I don't necessarily need to be informed of something that is going wrong in the other state. If I'm not responsible for that state, definitely this needs to be filtered out. And also, I need to make sure that what I can get to is just one or two clicks away. And also, it’s definitely important to think about, if I want the data flow to be in both directions. As I said, there are a lot of benefits to getting information from the remote worker, just as it is to inform the remote worker. So, you need to think about how they would put information into the system. How are they contributing their knowledge to the overall monitoring of equipment, and of course, security. Please don't forget security. It’s not something you need to be afraid of, you just need to know that it's something you should consider going into a solution. Particularly because most of these technologies are internet exposed today. Just know it's there and be ready to think about it.
[37:53]
So, with that, unfortunately, Shane couldn't be with us today. But he was nice enough to give us a recording ahead of time. I spoke with Shane Stevens from Impact Automation a little while ago. He is one of the head integrators that's been working on the Lake Cities Municipal Utility Authority in Texas just north of Dallas. So, we've got a prerecorded talk we're going to play for us right now.
[38:16] Mr. Shane Stevens, Owner of Impact Automation and Controls
I am Shane Stevens with Impact Automation and Controls, and I'm happy to be part of ICONICS Connect 2021. Impact Automation and Controls provides consulting, configurations, programming, troubleshooting, and training services for industrial and municipal customers. We're located in the Dallas-Fort Worth area; we are able to work remote or onsite all over the United States. Today we're going to talk about Lake City's Municipal Utility Authority otherwise known as LCMUA. They’re located in Lake Dallas, Texas. They provide superior drinking water, fire protection and pressure, and sewage collection for the Tri Cities area - Shady Shores, Lake Dallas, and Hickory Creek. They operate 21 lift stations, 46 pumps, three pump stations with 10 pumps, three elevated storage towers, and three ground storage tanks.
[39:20]
LCMUA as an onsite solution to provide visibility to the team being that they run a lean operating system; they don't always have an operator 24/7 to monitor the system. To accommodate that, Lake Cities’ often remote workforce LCMUA uses a remote solution to extend their SCADA to their smartphones. This allows them to have access to SCADA data wherever they go.
[39:46]
The team always has access to the system for their mobile devices. So why send automated notifications for alarms? Well, they're typically busy working on projects, preventative maintenance, or fixing repairs, leaks, things like that. They can’t always watch the operations at all times. By sending them text messages to their smartphones, they can only be notified when they need to have their attention pulled away from their daily duties. This allows them to be more efficient, while maintaining a leaner workforce. As the message comes in on your smartphone, they've got their device in their hand. They can quickly switch over to the mobile app and see what's going on acknowledge alarms, send out a truck, whatever they need to do to get that problem solved. They can make that decision very quickly.
[40:40]
They also needed a system that allows them to share the workload; with the solution they chose allows them to share the workload effectively. We can schedule it, so we're sending water alarms to water department, wastewater alarms to the wastewater department. Rather than everyone getting the same alarms all the time, we can decide who gets what alarm when.
[41:03]
Scheduling is very important as operators are not always on shift. They may be on vacation. We've got on call rotations we need to change. And with the new hires and retiring operators, we need to be able to maintain that efficiently. So, their solution is simple to adjust. And we can change the schedule as needed to make sure we're running at optimum efficiency.
[41:33]
What about security? That's what everybody asks. We've got operators out in the field with mobile devices access in the SCADA system. How are we keeping that secure? Lake Cities thought this, so they hired an outside IT consulting group to develop a security model that'll fit their needs. Utilizing DMC and end to end encryption, they've secured it in a way that suits them. Their goal was to extend the network to support their mobile workforce without sacrificing security constraints. And I believe they accomplish that. Again, this is Shane Stevens with Impact Automation and Controls and I'm happy to be part of ICONICS Connect 2021. Thank you.
[42:19] Mr. Jotham Kildea ICONICS Solution Sales Supervisor
Thanks to Shane. Again, I'm sorry you couldn't be here, but we were able to capture some of his insights on that ahead of time. I’m going to switch over now. Shane was actually gracious enough to give me access just for the week to their actual system. So, I pull it up on an iPad. So, I set up with this system so that if we get an alarm, something goes wrong in the system. Okay, turn on an alarm; it's actually going to send me a text message. I'm probably not going to be able to hear it too well, for you guys, unfortunately, but, you know, sends me a text message just says, “Hey, this is a situation for their workers”. That's the typical workflow that they'd get a text message or an email pop up that says, hey, something's wrong. You know, they know working part of their practice and procedure, they're going to go to the MobileHMI and check and take a look at it; they might use a visual like this. So, this is just the standard alarm viewer, they can see what's going on. And the system, I don't know much at all about that second alarm because that's something going on today. But let's say that they get a message. If they need to take action, they could just directly from this acknowledge. This is that acknowledgement response I was talking about. This is also MobileHMI. So, if I go back, we can also look at some of the other things within the application.
[43:51]
So, there are a few different systems; they're both a freshwater and wastewater treatment system. So, I'm going to the water system, and see some of the graphics here. This is something a user might do before responding and acknowledging to an alarm to figure out a little bit more nuance what's going on in the system. They can browse into the system to see what's going on. I have read only access and, and for good reason. I don't know what I'm doing but they'd be able to go in here, perhaps take control of certain situations or just see what's gone on. If they're dissatisfied with what's going on, they can just go back to the alarm viewer, and then they can go ahead and acknowledge that alarm.
[44:31]
Let's see. I'm going to browse back to that alarm viewer. And straightforward, just click on that alarm. And I can acknowledge it directly from this viewer; this is pretty much what they do day in and day out to actually operate the system. So that was pretty cool. He's nice enough to give me access to this and trusted me not to acknowledge that other stuff. That's kind of what they use that thing for. It's not complicated, not crazy unusual how they set it up; this is something that they can take with them out in the field. And hey, something's gone on, they get a message. Oh, I can take a look at it; maybe they may just dismiss it. Maybe they may take a look and figure out what they're doing and acknowledge. Simple as that. So that's what I'm going to share with you. I'm also going to pull up Josh at this point. And we can have a look at some of the other applications we've put together. I definitely want to show you Shane because real applications are always king, but I want to show you some of the other stuff we've been playing around with for design and how users are interacting system.
[45:38] Mr. Joshua Obal ICONICSS Senior Software Engineering Supervisor
I put together a demo today so I could give everybody an idea of a lot of those different capabilities that Jotham was mentioning. During the slides, we're going to go through these analytics and dispatching dashboard. I'm going to show a little bit of work order integration, integration with Microsoft Teams, some of the intelligent alarm escalation, workflows, things like that. So, to get started here, this is the home screen on the analytics and dispatching dashboard. You can see some key metrics here, like the number of active alarms and unacknowledged alarms, and the lists the current alarm list. We have some details about how workers have been responding to the alarms and actual responses that they've given to some of the alarms. CFSWorX has some additional capabilities where you can accept or reject an alarm or say you're busy. And when I say alarm, I mean events in the general term, like Jotham was saying, could be a fault or other alarm source. And then on the bottom section here is some information about the geofences and worker location like Jotham was mentioning.
[46:59]
I'm going to switch over to the dispatching page here. I want to show an example. This is in the West Texas area. We're going to do a demonstration where Jotham will fill the role of the field worker, and I'll be the dispatcher. On the left side here there's an expandable collapsible tree control which shows our assets in the system. I can right click and zoom in. And in the map control in the middle, we can see where all of the different assets are located, where all the workers are located. So Jotham is pretty close to this pump station here. I can zoom out a little bit; I can pan down. I'm located here in the Andrews area in the control room. And I'm doing the dispatching. So, when an alarm comes in, we're going to be able to click on it in the dashboard to see what the details are of that alarm, where it's located, the severity, and we can even see a list of the workers to see who is closest to the alarm. Also, we can see whether or not the person is on schedule, and where we are getting some additional information from the person's phone using our MobileHMI app. So, we get the location information, the battery, and the signal power. So, if you're manually talking to some of these workers, you can tell if you have a good chance of being able to get ahold of them on the phone. I can filter here by who's currently on schedule.
[48:49]
The scheduling information could be coming from one of those CRM systems that Jotham mentioned, or it can be defined locally in the ICONICS software. In the upcoming 10.97.1 release, we added this new page here for scheduling. So, if you're using the ICONICS scheduling system, you can go here, see the list of your workers, and you can click on one of them. You can bring up their schedule to view it. So, Janet Leverling is using the shared second shift schedule which could be used by multiple workers. You could alternatively give a schedule to each worker individually. So, you can assign different appointments or something in somebody's day. So here you can see that second shift schedule. It also has support for holidays and exceptions. And there's a runtime tab where you can go to see the current status of that person's availability. Let me go back to the dispatching page now. And I want to give a live demo of dispatching an alarm to the field worker who will be Jotham. Real quick, I'm going to show you what's going to happen. This is the workflow that we're using. So, the blue circle here represents where the incoming alarm will occur and be received in the system. Then there's a worker lookup activity; it's going to look for the field worker who's available. In this case, I'm just looking for Jotham, that provides his name, his contact information, his availability. The next activity here is going to send him an email via SendGrid. There'll be a small delay, and then it'll check to see if he acknowledged the alarm. If not, it could go on to the next field worker in the system and notify the next person. In this case, I want to go forward and escalate that alarm to the dispatcher to let me know that he didn't acknowledge it in time. So let me go back to the dispatcher page. And I'm going to activate this alarm right near Jotham. And I'll switch over to his mobile device view. So, we can see what he's seeing on his phone. Now, he's going to be getting the email, and I'll let him take over,
[51:32] Jotham Kildea
We set up a screen share, so you don't have to squint while holding it up. So as Josh mentioned, I actually just got that notification. So, you can see that on the top of my list here. This is in Outlook. This is the message that came from SendGrid. If I look at it, everything's configurable in the format of the message. So, if I wanted to, there's instructions in here of how I could acknowledge it by putting in a special acknowledge ID to just acknowledge it straight out. But of course, I want to see a little bit more detail. So, I'm going to use the link here to jump to figure out what's going on with that system.
[52:04]
So, click on the link, and this is going to bring me to the view of that alarm. So already I've got the alarm synopsis a little bit better formatting of what's going on in that alarm. And across the bottom, you'll see that there's quite a few different things that I could do to take action. When we set up this demo, I told Josh, “Hey it’s a Connected Worker session. We got to talk about how we connect to workers.” So, there are a few different things you can do here. One of the simple things, of course, is we know how to do this very well, Equipment Details, so fourth one down. This is going to take from the alarm detail to more of a traditional SCADA view. So, this is going to bring up information about the actual asset that is triggering that alarm. So, I'm just going to browse the system. This is pretty straightforward to see, “Hey, there's something wrong there with bar power.” I can look at that pretty plainly. But from a SCADA view like this, you might also look at prior maintenance logs, look at other related equipment upstream or downstream. You might look at what's going on with other systems. When did this problem happen? What preceding factor - all that historical data, all that fun stuff will be there. This is an example of how that might happen.
[53:11]
To go back, you can browse back to the alarm. Equipment Details, definitely one thing you might look at. Another thing you might look at would be creating that work order. So, Josh mentioned we’re integrated to dynamics here; that's going to be the second button here, create the work order pops up with the details, it filled in the summary of what's wrong with the information just based on the alarm that could be coming from other attributes of the alarm, whatever you want. I'm going to just put some quick instructions here. Let's see. Okay, place filter, go ahead and create a work order. So, what that's going to do is go off and actually integrate that and put that into the dynamic system. So actually, if I go to view the work order, that's going to take me over to one of those multi-apps versus one app switches. We've now just launched over to the Dynamics 365 Interface; this is on the web interface. So, I can see this work order that we just created with the information that I put in about the pump. The summary is on the second page and there are my instructions in my replace filter. So, all that information is going to be plugged right into the work order might be a situation that you have as the eventuality of what do you do with this alarm? So, let's see what I've created. I've looked at some of the information. I've created a work order for it. Maybe before I go to the site, I want to talk to the dispatcher, Josh, to say, “Hey, I got more questions”. So, I'm going to contact dispatch on the bottom; this is going to bring up Teams.
[54:44]
So, this is going to jump to the Teams app. It's already pointed me to a chat with Josh. We might want to take a shortcut and say, “Hey, you might want to use this information.” So, let's just pre-populate that field. I can delete it. But let's say I say, “When was this last serviced?” Grandson that that's going to send a Teams message to Josh. Nothing unusual there. That's just standard Teams message. So, I'm in the Teams chat. He can see it on his device. There it is; it popped up. Okay, send me a message. Already got that here. Simple as that. We're in the same room. Obviously, I'd be out in the field; he'd be back on the site. It's Teams. So of course, that was just a quick text message I might send, start an audio, or video call with him, just standard features there. Just so you know how to keep people connected. Because hopefully information is right there on the screen for me. But if not, everything's there, or there's some stuff that's not in the system. Or I say, “Hey, Josh, why'd you send this to me?” I want to call them up and ask. So it’s very useful to have that integration there. So let me jump back to the screen one more time. So back here.
[55:53]
As I said, I obviously can acknowledge the alarm. One thing I wanted to show is just getting the directions as another thing. So, it's actually in the alarm view. I just have the directions; click here. If I've decided I'm going to take over this alarm, or work order, I'm going to figure out where I want to go. One part of the demo, we didn't manage to fake or didn't manage to sync up, right? I'm in Massachusetts, the assets in Texas. So, it's a very long drive for me. I'm not actually driving that. That's one of the eventualities of that, so it will bring up the again external apps. We'd like to think it, but we're not the best at navigation and mapping, Google Maps and Bing Maps are pretty good. So just use those. So, it's going to bring up the maps and tell you how to get there. And the last thing is I acknowledge it, that's honestly the easiest part of it. Just do a quick acknowledgement. I can see. I took ownership of it. I acted as a mobile user.
[56:49] Josh Opal
Alright, I have a few more minutes here. I'll show you. You saw I had a 15 second delay in my workflow earlier. And Jotham didn't acknowledge it within that allotted 15 seconds, I also got a message in my email that said there was an unacknowledged alarm on pump station. So that gives you an idea of the of the escalation scenario where the field worker, or field workers all get notified about the alarm. If they don't reply, then you can let the manager or dispatcher know that there's something more important that needs to be looked at. And you have full control to customize those workflows.
[57:39]
A couple more things in the analytics dashboard. There's the event Analysis page where you can see how all the workers have been responding to the alarms in your system. So, there's different statistics being calculated here by our AnalytiX-BI server, telling you average response time in minutes, average response time by asset source, how many responses each worker has done. And different response types per worker. So, you know that somebody might be rejecting everything. Maybe they're busy right at that time, and you need to talk to them about what they're working on because they've been rejecting everything.
[58:37]
There's the geofencing. So, as you saw in the dispatching page. We had little shapes drawn around some of the assets. And like Jotham explained earlier, we might want to know when somebody is getting close to a particular asset, so they can do some work. Some maintenance work on that. I can zoom in and out on the map here. And this is showing some of the history of how many times different workers are crossing in and out of different geofences. How long they were in a particular geofence. And then all of those geofence events are shown on the bottom here.
[59:24]
There's a page for the live worker information. Again, you can either be getting the location information from our MobileHMI app, or we do have support now to import the location information from third party apps like Salesforce, ServiceNow, things like that. But this will, this will tell you again, the geofence events that are occurring where everyone is located, how long they've been in a particular geofence.
[1:00:01]
And then the final page here is the ICONICS remote expert technology which would allow you to initiate a call from a worker to the dispatcher to get additional assistance on resolving an issue. So that is everything. Thanks, Jotham. Thank you, Josh.
[1:00:28]
I want to give a quick breakdown on how to connect remote workers. Where do to get started? Some things to keep in mind is think about what do you want the experience? What is the interface supposed to be like? Right? Different people have different needs out of their jobs. Some are going to be busier with their hands all day. Other times they can access the thing while they're in the car, but not when they go to the site. What sort of information do they need to make the decisions? And what sort of decisions should be made from the field versus what should be made on site? Think about how remote workers are going to interact with the data. Plan for deployment in advance; think about what you want to do for security considerations. And you want to design the system?
[1:01:20]
Before you start, of course, and also consult with users. I can't stress enough that a successful project will take into consideration what the actual users and the operators are expecting or want to see out of the system; you may have misunderstood what they actually need out in the field. So, talk to them; see what they want to do. And then for getting started, a lot of what we're talking about for connectivity for remote workers can be built and layered upon existing systems. We're not talking about throwing out and reinventing the wheel here. These are enhancements to solutions that are already there today but just bring them out to the people that are wherever they are. Pilot applications are a good way to get started. Take a portion of the workforce and say, “Hey, let's try this out for a few months to see if this going to work?” Let's take that feedback and redesign for the eventual full rollout. And always plan big but think small as far as what you actually want to do and what steps you can take to get that going?
[1:02:23]
Megan Courtney is going to join us, and we're going to feel some of the questions that came in again, that's on the Attendee hub. Thank you.
[1:0234] Ms. Megan Courtney ICONIS Marketing Coordinator and Social Media Specialist
Hello, everybody. Jotham. Josh, fantastic presentation, as usual. So, we did receive a few questions. The first one: “Is remote acknowledgement of notifications always tied to a mobile device or can it be handled independently?”
[1:02:51] Jotham Kildea
Oh, sure. So, for remote notification, anybody not in a traditional SCADA screen environment absolutely can be handled through other things, for example as I mentioned, SMS. There's an acknowledge code that as a user, you can plug it in and just respond to an alarm. And that handles an acknowledgement. Same thing effectively for voice and email. Even you can do the same thing as far as emailing responses. So that frequently can be a quick thing. It doesn't always involve the mobile interface. A lot of times, applications just go the route of “Hey, I got a text message. I can respond in just quick few seconds”.
[1:03:26] Megan Courtney
Right. Our second question that we received is “What do we have to be concerned about if we have to comply with GDPR?”
[103:34] Jotham Kildea
Yeah, so I mentioned this a little bit in the session, but GDPR is very concerned with making sure that personally identifiable information is that they've opted in or authorized the use of that information, and it can be removed from the system at will. So, a lot of what CFS WorX is doing is a kind of an anonymized actor ID that says, “Here's a generic user”. It doesn't actually personally identify people. We use that for a lot of transactional information of “who do we contact and how”. So that is separable from the personal information because a lot of times when you're deploying a system, if you're configuring it, you want to know, “Okay, I'm sending a message to Josh, not user 1278”. But under the hood, those things are entirely separable. And that's a good way to make sure that if you decide to remove, for example Josh, you remove his system, it's going to be decoupled from any personally identifiable information.
[1:04:28] Megan Courtney
All right, our next question, “Does CFS WorX require that there's some dispatcher making the decisions about who to notify?”
[1”04:37] Jotham Kildea
No, no. So, in the demo that we showed you that Josh was running, we used the example of a dispatcher dashboard. That's absolutely one way we see some applications go where they have somebody that's actually making the decisions of, “Hey, this notification is going out to this person.” You can go that route. You can also very commonly go with the automated dispatch with no humans in the middle; it just sends notices out to a predefined queue of workflow of users. For example, Lake Cities do that since they don't have a lot of in-person staff that are on site all the time. They mostly have remote workers, so they just automatically dispatched to who needs to be informed.
[1:05:21] Megan Courtney
Awesome. Next one, “Can we get worker location data from other sources if there are privacy limitations to getting it directly from an app?”
[1:05:29] Jotham Kildea
So, privacy, do you want to feel that, Josh?
[1:95:35] Josh Opal
What I mentioned in the demo was that you can get the “Locate Worker Location” from the ICONICS MobileHMI app. There are also third-party apps. But typically, we're going to be getting the worker location from their cell phones which might be running the ICONICS MobileHMI app or a Dynamics or Salesforce app that's communicating back to their server. So there has to be something collecting information about where the worker is located, transmitting it back to some server so that our server can make the determination of who's closest and who needs to receive the notifications.
[1:06:24] Jotham Kildea
And to be clear: That the MobileHMI is an opted-in type of deployment for that. It’s not secretly tracking where you are. It's something you've configured and enabled to expose that information for the system.
[1:06:36] Megan Courtney
So, looks like we have one more question, “You mentioned Managing Schedules - is that something that's done on a per user or group basis?”
[1:06:45] Jotham Kildea
It depends. So, Josh showed a little bit of the dashboard where you can configure the schedule environment from that space, and it really depends on the application. Some folks if it's a smaller system, you might just say, “Hey I've got eight workers. I can configure everybody's schedule. I know when they go on holidays.” it's individualized. Or you might have a broader model where it's kind of like a first shift, second shift or who's on call at certain times. You can go either way. Obviously, a group schedule is going to be easier to shift and adjust at scale, but the personalized approach of individualized schedules really gives you that free control. As I said, I typically see that with smaller applications where it’s much more of a personal interaction where they're going to say, “Yeah, I can cover that shift, or I can do these different times or someone's on vacation”. And you can adjust it very finitely.
[1:07:36] Megan Courtney
That was it for the questions. Thank you.