Agent¶

das-Peak Streaming Twilio agent based solution is built as a Twilio Flex Plugin. It customizes Twilio Flex UI giving agents the ability to register/verify user's voice. This plugin is designed to use Veridas das-Peak Streaming service within a Twilio environment in an easy way.

Twilio Flex Plugin

Integration steps¶

Next steps will show you how to create and configure a complete integration. As a previous requirement, customers need to have a Twilio account and some administration knowledge.

Step1: Get Javascript files from Veridas¶

Veridas provides some Javascript files with all required logic to integrate with das-Peak Streaming. Following files are provided:

daspeak-streaming-web-agent-twilio.bundle.js: Containts all integration logic and UI implementation.
config.js: Includes integration configuration.
utils.js: Includes some utilities such as database storage and authentication.

Contact with Veridas to get das-Peak Streaming Twilio integration files.

Step2: Create Twilio Flex Plugin¶

Check how to create a Twilio Flex instance and how to develop your own Twilio Flex Plugin on Twilio's official documentation.

Step3: Create Veridas Component¶

Create a src/components/veridas/ folder and add a VeridasComponent.js file inside it, with the following content:

import React from 'react';
import { TaskHelper, withTaskContext } from '@twilio/flex-ui';

import './daspeak-streaming-web-agent-twilio.bundle.js'
import config from './Config.js';
import utils from './Utils.js';


class VeridasComponentAux extends React.Component {
    component = null
    constructor(props) {
        super(props);
        this.referenceComponent = React.createRef();
    }

    componentDidMount(){
        const component = this.referenceComponent.current;
        component.config = config;
        component.utils = utils;
    }

    render(){
        const { task } = this.props;
        let isLiveCall = TaskHelper.isLiveCall(task).toString();
        let conferenceSid = null;

        if (typeof task.conference !== 'undefined') {
            if("conferenceSid" in task.conference){
                conferenceSid = task.conference.conferenceSid;
            }
        }

        return (
            <daspeak-streaming-web-agent-twilio 
                ref={this.referenceComponent}
                securityToken={this.props.token}
                callSid={task.attributes.call_sid}
                callerId={task.attributes.caller} 
                taskSid={task.taskSid}
                conferenceSid={conferenceSid}
                callInProgress={isLiveCall}/>
        );
    }
}

export default withTaskContext(VeridasComponentAux);

Veridas Component contains all das-Peak Streaming Integration required logic and UI management. Check how daspeak-streaming-web-agent is imported and used within Veridas Component.

Veridas Component uses several Twilio Flex parameters to work. DO NOT EDIT THEM or the plugin will not work as expected. The following parameters shall not change:

securityToken: It is a security measure recommended by Twilio documentation for calling Twilio functions from Flex plugins.
callSid: Current call identifier.
callerId: Customer telephone number.
taskSid: Current flex task identifier.
conferenceSid: Unique identifier of the current conference.
callInProgress: bool value, true if call is active.

Additionally, inside the src/components/veridas/ folder is where you must put the files that you obtained in step 1.

Step4: Set configuration and utilities file¶

config.js and utils.js file should be filled with suitable values for integration. Make sure to set these values and to store these files before running the integration.

Next, an example of a configuration file is shown:

export default {
  daspeakStreamingSignallingUrl: '{{ DASPEAK_STREAMING_SIGNALLING_URL }}',
  daspeakStreamingAudioUrl: '{{ DASPEAK_STREAMING_AUDIO_URL }}',
  authenticationServerUrl: '{{ GET_TEMPORARY_TOKEN_TWILIO_FUNCTION_URL }}',
  urlStartStreaming: '{{ START_STREAMING_TWILIO_FUNCTION_URL }}',
  urlStopStreaming: '{{ STOP_STREAMING_TWILIO_FUNCTION_URL }}',
  register: {
    audioTotalDuration: '5',
    authenticityThreshold: '0.8',
    modelHash: '{{ MODEL_HASH }}'
  },
  verify: {
    audioTotalDuration: '3',
    authenticityThreshold: '0.8',
    scoreThreshold: '0.8'
  }
};

Configuration file must be filled with suitable values:

Parameter	Description	Allowed values
daspeakStreamingSignallingUrl	das-Peak Streaming Socket.IO Signalling Channel URL (Provided by Veridas)	URL format
daspeakStreamingAudioUrl	das-Peak Streaming Twilio Audio URL (Provided by Veridas)	URL format
authenticationServerUrl	URL of authentication backend to obtain of twilio function.	URL format
urlStartStreaming	URL of start streaming to obtain of twilio function.	URL format
urlStopStreaming	URL of stop streaming to obtain of twilio function.	URL format
register.audioTotalDuration	Minimum amount of seconds of speech required to process the audio for a registration process.	['0.0', '30.0']
register.authenticityThreshold	Authenticity threshold required for a registration process to succeed.	['0.0', '1.0']
register.modelHash	Hash of biometric model to be used.	(Provided by Veridas)
verify.audioTotalDuration	Minimum amount of seconds of speech required to process the audio for a verification process.	['0.0', '30.0']
verify.authenticityThreshold	Authenticity threshold required for a verification process to succeed.	['0.0', '1.0']
verify.scoreThreshold	Minimum threshold required for a verification process to succeed.	['0.0', '1.0']

Check how to configure utilities file.

Step5: Use VeridasComponent from created Plugin¶

When the Twilio Flex Plugin project was created, there is a .js file inside the src folder, which uses the FlexPlugin class, inside this file we must do the following:

Import Veridas Component inside the Twilio Flex Plugin project like this:

import VeridasComponent from './components/veridas/VeridasComponent.js';

Lastly, index Veridas Component into init function. You can use the following snippet:

flex.CallCanvas.Content.add(<VeridasComponent key="veridas-component" token={manager.store.getState().flex.session.ssoTokenPayload.token}/>, { sortOrder: 1 });

Step6: Create Veridas Service with needed functions¶

To start using Twilio functionality, it is required to add some code on Twilio console. To proceed:

Go to Twilio Console
Click on Explore products on the left panel
Go to Runtime section and click on Functions and Assets
Click on Services > Create service
Set up name field VeridasService and click on Next
Click on Add > Add Function and set /startStreamAudio as name
Click on Add > Add Function and set /stopStreamAudio as name
Click on Add > Add Function and set /getTemporaryToken as name
Insert code snippets from below sections
Make sure your functions are set as public (protected by default)
Click on Settings & More > Dependencies on the bottom left of the page
Add twilio-flex-token-validator module (version 1.5.5)
Add form-data module (version 4.0.0)
Add axios module (version 0.23.0)
Add https module (version 1.0.0)
Add util module (version 0.11.0)
Add lodash module (version 4.17.11)
Add fs module (version 0.0.1-security)
Add xmldom module (version 0.1.27)
Update twilio module version to 3.77.2
Update @twilio/runtime-handler module version to 1.0.1
Click on Settings & More > Environmental Variables
Set client_id, client_secret and getTemporaryTokenCloudUrl variables (values provided by Veridas)
Save and deploy the new functions

Note: An already existing Service can be used instead of a new one. You can also use the Twilio Serverless Toolkit to develop these functions locally and deployment as part of a pipeline.

Created functions will be called from Veridas Plugin to start audio streaming, stop audio streaming and get authentication token.

Start Stream function¶

This function starts streaming audio to das-Peak Streaming using Twilio Media Streams Audio Channel.

Open the "startStreamAudio" function and paste the following block of code. Save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

async function requestStartStream(context, event, callback, response) {
    const twilioClient = context.getTwilioClient();

    const callSid = event.callSid;
    const url = event.url;

    twilioClient.calls(callSid)
    .streams
    .create({
        name: callSid,
        url
    })
    .then((stream) => {
        response.body = JSON.stringify({'streamSid': stream.sid});
        return callback(null, response);
    })
    .catch((error) => {
        console.error(error);
        return callback(error);
    });
}

exports.handler = TokenValidator(function(context, event, callback) {
    const response = new Twilio.Response();
    response.appendHeader('Access-Control-Allow-Origin', '*');
    response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
    response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');

    requestStartStream(context, event, callback, response);
});

Stop Stream function¶

This function stops audio streaming to das-Peak Streaming.

Open the "stopStreamAudio" function and paste the following block of code. Save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

async function requestStopStream(context, event, callback, response) {
    const client = context.getTwilioClient();

    const callSid = event.callSid;
    const streamSid = event.streamSid;

    client.calls(callSid)
    .streams(streamSid)
    .update({status: 'stopped'})
    .then((stream) => {
        return callback(null, response);
    })
    .catch((error) => {
        console.error(error);
        return callback(error);
    });
}

exports.handler = TokenValidator(function(context, event, callback) {
    const response = new Twilio.Response();
    response.appendHeader('Access-Control-Allow-Origin', '*');
    response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
    response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');

    requestStopStream(context, event, callback, response);
});

Get Temporary Token function¶

This function calls Veridas Authentication Service in order to get a valid authentication token to use das-Peak Streaming.

Open the "getTemporaryToken" function and paste the following block of code. Save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

function generateCloudRequest(context) {
  const FormData = require('form-data');
  const https = require('https');

  const formdata = new FormData();
  formdata.append("grant_type", "client_credentials");
  formdata.append("client_id", context.client_id);
  formdata.append("client_secret", context.client_secret);

  const requestConfig = {
    method: 'POST',
    url: context.getTemporaryTokenCloudUrl,
    data: formdata,
    headers: { 
    ...formdata.getHeaders()
    },
    httpsAgent: new https.Agent({
      rejectUnauthorized: false
    })
  };

  return requestConfig
}

exports.handler = TokenValidator(function(context, event, callback) {
  const axios = require("axios");
  const response = new Twilio.Response();
  response.appendHeader('Access-Control-Allow-Origin', '*');
  response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
  response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');
  response.appendHeader('Content-Type', 'application/json')

  const requestConfig = generateCloudRequest(context);
  axios(requestConfig)
  .then(function (result) {
    response.setBody(`${result.data.token_type} ${result.data.access_token}`);
    return callback(null, response);
  })
  .catch(function (error) {
    console.error(error);
    return callback(null, error);
  });
});

Step7: Configure Veridas Plugin¶

Please visit the following website to get in touch with Veridas.

Veridas Temporary Token url¶

Once Veridas provides you with valid credentials and configuration urls, enter the Environment Variables section within the previous window (Twilio Functions and Assets), and insert the following one with the provided info:

getTemporaryTokenCloudUrl: URL to the Veridas endpoint to obtain a temporary token.

Veridas credentials¶

In the same Environment Variables section within the window (Twilio Functions and Assets), set the following credentials provided by Veridas:

client_secret
client_id

Twilio Streaming functions urls¶

In the previous window (Twilio Functions and Assets), you can obtain the urls to execute the new functions. Click on the three dots button next to each function name and copy the url.

Set these urls in the Config.js file attached above. Parameters to set:

urlGetTemporaryToken: URL to your getTemporaryToken Twilio Function
urlStartStreaming: URL to your startStream Twilio Function
urlStopStreaming: URL to your stopStream Twilio Function

Step8: Deploy and test Pugin¶

Follow Twilio's instructions to deploy plugins and use them inside Twilio Flex.