实时语音翻译 API 简介

概念解释

实时语音翻译：可实现对连续音频流的实时翻译和整句翻译，转换成识别后文本信息并返翻译后文字流。

说明

Hi，您好，欢迎使用有道智云实时语音翻译API接口服务。

如果您想快速体验服务，建议您前往 实时语音翻译体验中心 或者在体验中心右下侧找到小程序二维码，扫描进行体验。

如果您有与我们商务合作的需求，可以通过以下方式联系我们：

商务邮箱： AIcloud_Business@corp.youdao.com

如果您对文档内容有任何疑问，可以通过以下几种方式联系我们：

客服QQ：1906538062

智云翻译技术交流QQ 1群: 652880659

智云翻译技术交流QQ 2群: 669384425

智云翻译技术交流QQ 3群: 807539209

智云翻译技术交流QQ 4群: 936752411

联系邮箱： zhiyun@corp.youdao.com

温馨提示：

本文档主要针对开发人员，接入测试前需要获取应用ID和应用密钥；如果您还没有，请按照 新手指南 操作。

平台向每个账户赠送50元的体验金，供用户集成前测试所用，具体资费规则详见 实时语音翻译服务报价 。

接口说明

接口地址：

wss://openapi.youdao.com/stream_speech_trans?{请求参数}

协议须知

调用方在集成实时语音翻译API时，请遵循以下规则。

接口调用参数

服务接口的调用分为认证、实时通信两阶段。

1 .认证阶段

请求参数格式：
key1=value1&key2=value2&key3=value3&key4=value4

参数说明：

• 注意：noitn参数只适用于以下语种：中文（ zh-CHS ），粤语（yue ），英文（ en ），英语(澳大利亚)（ en-AUS ），英语(印度)（en-IND ），英语(英国)（en-GBR ），日文（ ja ），韩文（ ko ），西班牙语（ es ），俄语（ru ），阿拉伯语(ar ）

签名生成方法如下：
sign=sha256(应用ID+salt+curtime+应用密钥)。（仅API接入方式的应用有应用密钥）

2. 实时通信阶段

认证成功之后，进入实时通信阶段，此阶段客户端发送音频流和结束标识，并接收转写结果。

2. 2. 1 发送音频流

此阶段客户端通过 binary message 发送音频流，内容为音频的二进制数据，此过程的发送频率将影响文字结果展示的实时性。

建议以 200ms 间隔发送音频数据，若间隔超时 15s 以上，服务端将停止识别。

2. 2. 2 发送结束标识

客户端完成所有音频数据的发送后，需发送一个特殊的 binary message 到服务端作为音频流
发送结束的标识，内容为：

{"end": "true"}

输出结果

认证结果说明

服务端通过 text message 返回 json字符串的认证结果，参数示例：

成功：

{
    "result": {},
    "action": "started",
    "errorCode": "0"
}

失败：

{
    "result": "{}",
    "action": "error",
    "errorCode": "202"
}

识别结果说明

交互过程中，服务端不断通过 text message 返回实时识别结果到客户端，响应结果是以json形式输出（为text message）。

识别结果 result 参数说明：

识别结果示例：

{
    "result": {
        "transPattern": "stream",
        "segId": 0,
        "bg": 60,
        "context": "Have a good day.",
        "tranContent": "祝你今天愉快。",
        "partial": true,
        "ed": 840
    },
    "errorCode": "0",
    "action": "recognition"
}

支持的语言表

• 注：目标语言不支持 “enzh 中英混合”

服务配置

错误代码列表

版本更新记录

常用语言Demo

Java 示例

package com.youdao.ai;

import javax.websocket.ContainerProvider;
import javax.websocket.Session;
import javax.websocket.WebSocketContainer;
import java.io.FileInputStream;
import java.io.InputStream;
import java.net.URI;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.UUID;

public class StreamSpeechDemo {

    public Session session;

    public static void main(String[] args) throws Exception {

        String filePath = "音频文件";
        String from = "源语言";
        String to = "目标语言";
        String appKey = "您的应用ID";
        String appSecret = "您的应用密钥";
        streamSpeech(appKey,appSecret,filePath, from, to);
    }

    protected void start(String uri) {
        WebSocketContainer container = ContainerProvider.getWebSocketContainer();

        try {
            session = container.connectToServer(Websocket.class, URI.create(uri));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void doAsrWebSocketClient(String filePath, String uri, Integer step) {
        StreamSpeechDemo streamSpeechDemoWebSocketClientApp = new StreamSpeechDemo();
        streamSpeechDemoWebSocketClientApp.start(uri);
        try {
            InputStream inputStream = new FileInputStream(filePath);
            int read = 0;
            byte[] bytes = new byte[step];
            while ((read = inputStream.read(bytes)) != -1) {
                streamSpeechDemoWebSocketClientApp.session.getBasicRemote().sendBinary(ByteBuffer.wrap(bytes));
                Thread.sleep(100);
            }
            byte[] closebytes = "{\"end\": \"true\"}".getBytes();
            streamSpeechDemoWebSocketClientApp.session.getBasicRemote().sendBinary(ByteBuffer.wrap(closebytes));


        } catch (Exception e) {
            e.printStackTrace();
        }
    }


    private static void streamSpeech(String appKey,String appSecret,String filePath,String from, String to) throws NoSuchAlgorithmException {
        StreamSpeechDemo asrWebSocketClient = new StreamSpeechDemo();
        String nonce = UUID.randomUUID().toString();
        String curtime = String.valueOf(System.currentTimeMillis()/1000);
        String signStr = appKey + nonce +curtime + appSecret;
        String sign =  encrypt(signStr,null);

        String uri = "wss://openapi.youdao.com/stream_speech_trans?appKey="+appKey+"&salt="+nonce+"&curtime="+curtime+"&sign="+sign+"&version=v1&channel=1&format=wav&signType=v4&rate=16000&from=" + from + "&to=" + to + "&transPattern=stream";
        asrWebSocketClient.doAsrWebSocketClient(filePath,uri,6400);
    }

    /**
     * 获取MessageDigest的加密结果
     * @param strSrc
     * @param encName
     * @return
     * @throws NoSuchAlgorithmException
     */
    public static String encrypt(String strSrc, String encName) throws NoSuchAlgorithmException {
        byte[] bt = strSrc.getBytes();
        if (encName == null || "".equals(encName)) {
            encName = "SHA-256";
        }
        MessageDigest md = MessageDigest.getInstance(encName);
        md.update(bt);
        // to HexString
        return bytes2Hex(md.digest());
    }
    public static String bytes2Hex(byte[] bts) {
        String des = "";
        String tmp;
        for (int i = 0; i < bts.length; i++) {
            tmp = (Integer.toHexString(bts[i] & 0xFF));
            if (tmp.length() == 1) {
                des += "0";
            }
            des += tmp;
        }
        return des;
    }
}


import javax.websocket.*;
import java.io.IOException;

@ClientEndpoint
public class Websocket {

    private void print(Object obj) {
        System.out.println(obj.toString());
    }

    Session session;

    @OnOpen
    public void onOpen(Session session) {
        print("Connect to endpoint: " + session.getBasicRemote());
        this.session = session;
    }

    @OnMessage
    public void onMessage(String message) throws IOException {
        print(message);
        if (message.contains("\"errorCode\":\"304\"")) {
            onClose();
        }

    }

    @OnError
    public void onError(Throwable throwable) {
        throwable.printStackTrace();
    }

    @OnClose
    public void onClose() throws IOException {
        if(this.session.isOpen()){
            this.session.close();
        }
        print("session close");
        System.exit(0);
    }

}

Python 示例

# -*- coding: utf-8 -*-
#!/usr/bin/python
import Queue
import argparse
import hashlib
import json
import sys
import threading
import time
import uuid

from ws4py.client.threadedclient import WebSocketClient

#sys.stdout = codecs.getwriter('utf-8')(sys.stdout)

app_key = '您的appKey'
app_secret = '您的应用密钥'


def rate_limited(maxPerSecond):
    minInterval = 1.0 / float(maxPerSecond)
    def decorate(func):
        lastTimeCalled = [0.0]
        def rate_limited_function(*args,**kargs):
            elapsed = time.clock() - lastTimeCalled[0]
            leftToWait = minInterval - elapsed
            if leftToWait>0:
                time.sleep(leftToWait)
            ret = func(*args,**kargs)
            lastTimeCalled[0] = time.clock()
            return ret
        return rate_limited_function
    return decorate


class MyClient(WebSocketClient):

    def __init__(self, audiofile, url, protocols=None, extensions=None, heartbeat_freq=None, byterate=32000,
                 save_adaptation_state_filename=None, send_adaptation_state_filename=None):
        super(MyClient, self).__init__(url, protocols, extensions, heartbeat_freq)
        self.final_hyps = []
        self.audiofile = audiofile
        self.byterate = byterate
        self.final_hyp_queue = Queue.Queue()
        self.save_adaptation_state_filename = save_adaptation_state_filename
        self.send_adaptation_state_filename = send_adaptation_state_filename

    @rate_limited(5)
    def send_data(self, data):
        self.send(data, binary=True)
        print 'data sent',len(data)

    def opened(self):
        def send_data_to_ws():
            if self.send_adaptation_state_filename is not None:
                print >> sys.stderr, "Sending adaptation state from %s" % self.send_adaptation_state_filename
                try:
                    adaptation_state_props = json.load(open(self.send_adaptation_state_filename, "r"))
                    self.send(json.dumps(dict(adaptation_state=adaptation_state_props)))
                except:
                    e = sys.exc_info()[0]
                    print >> sys.stderr, "Failed to send adaptation state: ",  e
            with self.audiofile as audiostream:
                for block in iter(lambda: audiostream.read(self.byterate/5), ""):
                    self.send_data(block)

            print >> sys.stderr, "Audio sent, now sending end message"
            self.send_data('{\"end\": \"true\"}')

        t = threading.Thread(target=send_data_to_ws)
        t.start()


    def received_message(self, m):
        print 'received',str(m)


    def get_full_hyp(self, timeout=60):
        return self.final_hyp_queue.get(timeout)

    def closed(self, code, reason=None):
        self.final_hyp_queue.put(" ".join(self.final_hyps))
def encrypt(signStr):
    hash = hashlib.sha256()
    hash.update(signStr.encode('utf-8'))
    return hash.hexdigest()

def main():

    parser = argparse.ArgumentParser(description='Command line client for kaldigstserver')
    parser.add_argument('-u', '--uri', default="wss://openapi.youdao.com/stream_speech_trans", dest="uri", help="Server websocket URI")
    parser.add_argument('-r', '--rate', default=32000, dest="rate", type=int, help="Rate in bytes/sec at which audio should be sent to the server. NB! For raw 16-bit audio it must be 2*samplerate!")
    parser.add_argument('--from', default="en", dest="lanFrom", help="en or zh-CHS")
    parser.add_argument('--to', default="en", dest="lanTo", help="en or zh-CHS")
    parser.add_argument('-p', '--pattern', default="sentence", dest="transPattern", help="The pattern of Stream Speech Translation should be sentence or stream")
    parser.add_argument('--save-adaptation-state', help="Save adaptation state to file")
    parser.add_argument('--send-adaptation-state', help="Send adaptation state from file")
    parser.add_argument('--content-type', default='', help="Use the specified content type (empty by default, for raw files the default is  audio/x-raw, layout=(string)interleaved, rate=(int), format=(string)S16LE, channels=(int)1")
    parser.add_argument('audiofile', help="Audio file to be sent to the server", type=argparse.FileType('rb'), default=sys.stdin)
    args = parser.parse_args()

    content_type = args.content_type
    if content_type == '' and args.audiofile.name.endswith(".raw"):
        content_type = "audio/x-raw, layout=(string)interleaved, rate=(int)%d, format=(string)S16LE, channels=(int)1" %(args.rate/2)

    nonce = str(uuid.uuid1())
    curtime = str(int(time.time()))
    signStr = app_key + nonce + curtime + app_secret
    print(signStr)
    sign = encrypt(signStr)

    ws = MyClient(args.audiofile, args.uri + '?appKey=' + app_key + "&salt=" + nonce + "&curtime="
                  + curtime + "&sign=" + sign + "&version=v1&channel=1&format=wav&signType=v4&rate=16000&from="
                  + args.lanFrom +"&to=" + args.lanTo + "&transPattern=" + args.transPattern, byterate=args.rate,
                  save_adaptation_state_filename=args.save_adaptation_state, send_adaptation_state_filename=args.send_adaptation_state)
    ws.connect()
    result = ws.get_full_hyp()
    print result.encode('utf-8')
if __name__ == "__main__":
    main()

C#示例

using System;
using System.IO;
using System.Net.WebSockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Security.Cryptography;

namespace asrliteclient
{
    public class AsrDemoWebSocketClient
    {
        public void start(String filePath, String uri, int step)
        {
            var client = new ClientWebSocket();
            client.ConnectAsync(new Uri(uri), CancellationToken.None).Wait();
            startReceiving(client);
            int bufferSize = step; //每次读取的字节数
            byte[] buffer = new byte[bufferSize];
            FileStream stream = null;
            try
            {
                stream = new FileStream(filePath, FileMode.Open);
                long fileLength = stream.Length; //文件流的长度
                int readCount = (int) Math.Ceiling((double) (fileLength / bufferSize)); //需要对文件读取的次数
                int tempCount = 0; //当前已经读取的次数
                do
                {
                    stream.Read(buffer, 0,
                        bufferSize); //分readCount次读取这个文件流，每次从上次读取的结束位置开始读取bufferSize个字节
                    //这里加入接收和处理数据的逻辑-
                    var array = new ArraySegment(buffer);
                    client.SendAsync(array, WebSocketMessageType.Binary, true, CancellationToken.None);
                    tempCount++;
                } while (tempCount < readCount);
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }
            finally
            {
                if (stream != null)
                    stream.Dispose();
            }

            try
            {
                byte[] closeBytes = Encoding.UTF8.GetBytes("{\"end\": \"true\"}");
                var closeArray = new ArraySegment(closeBytes);
                client.SendAsync(closeArray, WebSocketMessageType.Binary, true, CancellationToken.None);
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
                throw;
            }

            while (true)
            {
                Thread.Sleep(3000);
                break;
            }

            client.Dispose();
        }

        public async Task startReceiving(ClientWebSocket client)
        {
            while (true)
            {
                var array = new byte[4096];
                var result = await client.ReceiveAsync(new ArraySegment(array), CancellationToken.None);
                if (result.MessageType == WebSocketMessageType.Text)
                {
                    string msg = Encoding.UTF8.GetString(array, 0, result.Count);
                    Console.ForegroundColor = ConsoleColor.DarkBlue;
                    Console.WriteLine("--> {0}", msg);
                    Console.ForegroundColor = ConsoleColor.DarkGray;
                }
            }
        }

        public void doAscWebSocketClient(String filePath, String uri, int step)
        {
           start(filePath, uri, step);
        }

        public void init()
        {
            String filePath = "D:\\en.wav";
            String from = "en";
            String to = "zh-CHS";
            String appKey = "您的应用ID";
            String appSecret = "您的应用密钥";
            String nonce = Guid.NewGuid().ToString();
            TimeSpan ts = (DateTime.UtcNow - new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc));
            long millis = (long) ts.TotalMilliseconds;
            String curtime = Convert.ToString(millis / 1000);
            String signStr = appKey + nonce + curtime + appSecret;
            String sign = encrypt(signStr, null);

            String uri = "wss://openapi.youdao.com/stream_speech_trans?appKey=" + appKey + "&salt=" + nonce +
                         "&curtime=" + curtime + "&sign=" + sign +
                         "&version=v1&channel=1&format=wav&signType=v4&rate=16000&from=" + from + "&to=" + to + "&transPattern=stream";

            Console.WriteLine(uri);
            doAscWebSocketClient(filePath, uri, 6400);
        }

        //加密
        private String encrypt(String strSrc, String encName)
        {
            byte[] bt = Encoding.UTF8.GetBytes(strSrc);
            SHA256 sha256 = SHA256.Create();
            //暂时固定为SHA—256
            //暂时没发现内置的不同hash加密构造方法
            //可以自行编写代替
            if (encName == null || "".Equals(encName))
            {
                encName = "SHA-256";
            }

            byte[] HashData = sha256.ComputeHash(bt);
            StringBuilder oSb = new StringBuilder();

            for (int x = 0; x < HashData.Length; x++)
            {
                //hexadecimal string value
                oSb.Append(HashData[x].ToString("x2"));
            }

            return oSb.ToString();
        }

        public static void Main(string[] args)
        {

            AsrDemoWebSocketClient asrDemoWebSocketClient = new AsrDemoWebSocketClient();
            asrDemoWebSocketClient.init();
        }
    }
}

PHP 示例

<?php
# 需要安装Workerman库或用其他websocket第三方库代替

use Workerman\Worker;
use Workerman\Connection\AsyncTcpConnection;
use Workerman\Protocols\Websocket;
require_once 'C:\Users\谭上鸥\PhpstormProjects\Workerman-master\Autoloader.php';

function start($filePath,$uri,$step){
                     $worker = new Worker();

                     $worker->onWorkerStart = function($worker) use($uri,$filePath,$step){

                     $con = new AsyncTcpConnection($uri);

                     $con->websocketType = Websocket::BINARY_TYPE_ARRAYBUFFER;

                     $con->onConnect = function($con) {
        };

                     $con->onMessage = function($con, $data) {
            echo $data."\n";
        };

                     $con->connect();

                     $handle = fopen($filePath, "rb");
        // 每次读取一定大小的文件
                     $readTime = ceil(filesize($filePath)/$step);
                     $read = 0;
        while($read<$readTime){
                     $content = stream_get_contents($handle,$step,0);
                     $con->send($content);
                     $read++;
        }
                     $con->send("{\"end\": \"true\"}");
    };

    Worker::runAll();
}

//uuid generator
function create_guid(){
                     $microTime = microtime();
    list($a_dec, $a_sec) = explode(" ", $microTime);
                     $dec_hex = dechex($a_dec* 1000000);
                     $sec_hex = dechex($a_sec);
    ensure_length($dec_hex, 5);
    ensure_length($sec_hex, 6);
                     $guid = "";
                     $guid .= $dec_hex;
                     $guid .= create_guid_section(3);
                     $guid .= '-';
                     $guid .= create_guid_section(4);
                     $guid .= '-';
                     $guid .= create_guid_section(4);
                     $guid .= '-';
                     $guid .= create_guid_section(4);
                     $guid .= '-';
                     $guid .= $sec_hex;
                     $guid .= create_guid_section(6);
    return $guid;
}
function ensure_length(&$string, $length){
                     $strlen = strlen($string);
    if($strlen < $length)
    {
                     $string = str_pad($string,$length,"0");
    }
    else if($strlen > $length)
    {
                     $string = substr($string, 0, $length);
    }
}
function create_guid_section($characters){
                     $return = "";
    for($i=0; $i<$characters; $i++)
    {
                     $return .= dechex(mt_rand(0,15));
    }
    return $return;
}

function init(){
                     $filePath = "D:\\en.wav";
                     $from = "en";
                     $to = "zh-CHS";
                     $appKey = "您的应用ID";
                     $appSecret = "您的应用密钥";

                     $nonce = create_guid();
                     $curtime = strtotime("now");
                     $signStr = $appKey.$nonce.$curtime.$appSecret;
                     $sign = hash("sha256",$signStr);

                     $uri = "wss://openapi.youdao.com/stream_asropenapi?appKey=".$appKey."&salt=".$nonce."&curtime=".$curtime."&sign=".$sign."&version=v1&channel=1&format=wav&signType=v4&rate=16000&transPattern=stream&from=".$from."&to=".$to;

    echo $uri;

    start($filePath,$uri,1600);
}

init();
?>