android悬浮窗语音识别demo

1,347 阅读9分钟

转载请注明CSDN博文地址:blog.csdn.net/ls0609/arti…

在线听书demo:blog.csdn.net/ls0609/arti…
语音记账demo:blog.csdn.net/ls0609/arti…

Android桌面悬浮窗实现比较简单,本篇以一个语音识别,语义理解的demo来演示如何实现android悬浮窗。

1.悬浮窗效果

桌面上待机的时候,悬浮窗吸附在边上

这里写图片描述

拖动远离屏幕边缘时图标变大,松开自动跑到屏幕边缘,距离屏幕左右边缘靠近哪边吸附哪边

这里写图片描述

点击悬浮图标时,启动录音

这里写图片描述

说完后可以点击左button,上传录音给服务器等待处理返回结果

这里写图片描述

服务器返回结果后自动跳转到应用界面,本例用的是在线听书,跳转到在线听书的界面

这里写图片描述

2.FloatViewIdle与FloatViewIdleService

1.FloatViewIdle
定义一个FloatViewIdle类,如下是该类的单例模式

public static synchronized FloatViewIdle getInstance(Context context)
{
        if(floatViewManager == null)
        {
            mContext = context.getApplicationContext();;
            winManager = (WindowManager) 
                                    mContext.getSystemService(Context.WINDOW_SERVICE);
            displayWidth = winManager.getDefaultDisplay().getWidth();
            displayHeight = winManager.getDefaultDisplay().getHeight();
            floatViewManager = new FloatViewIdle();
        }
        return floatViewManager;
}

利用winManager 的addview方法,把自定义的floatview添加到屏幕中,那么就会在任何界面显示该floatview,然后再屏蔽非待机界面隐藏floatview,这样就只有待机显示悬浮窗了。

定义两个自定义view,分别是FloatIconView和FloatRecordView,前者就是待机看到的小icon图标,后者是点击这个icon图标后展示的录音的那个界面。

下面来看下怎么定义的FloatIconView

class FloatIconView extends LinearLayout{

        private int mWidth;
        private int mHeight;
        private int preX;
        private int preY;
        private int x;
        private int y;
        public boolean isMove;
        public boolean isMoveToEdge;    
        private FloatViewIdle manager;
        public ImageView imgv_icon_left;
        public ImageView imgv_icon_center;
        public ImageView imgv_icon_right;
        public int mWidthSide;

        public FloatIconView(Context context) {
            super(context);

            View view = LayoutInflater.from(mContext).
                                       inflate(R.layout.layout_floatview_icon, this);
            LinearLayout layout_content = 
                              (LinearLayout)  view.findViewById(R.id.layout_content);
            imgv_icon_left = (ImageView) view.findViewById(R.id.imgv_icon_left);
            imgv_icon_center = (ImageView) view.findViewById(R.id.imgv_icon_center);
            imgv_icon_right = (ImageView) view.findViewById(R.id.imgv_icon_right);
            imgv_icon_left.setVisibility(View.GONE);
            imgv_icon_center.setVisibility(View.GONE);

            mWidth = layout_content.getWidth();
            mHeight = layout_content.getHeight();
            if((mWidth == 0)||(mHeight == 0))
            {
                int temp = DensityUtil.dip2px(mContext, icon_width);
                mHeight = temp;
                icon_width_side_temp = DensityUtil.dip2px(mContext, icon_width_side);
                mWidth = icon_width_side_temp;
            }
            manager = FloatViewIdle.getInstance(mContext);
            if(params != null)
            {
                params.x = displayWidth - icon_width_side_temp;
                params.y = displayHeight/2;
            }
        }

        public int getFloatViewWidth()
        {
            return mWidth;
        }
        public int getFloatViewHeight()
        {
            return mHeight;
        }

        @Override
        public boolean onTouchEvent(MotionEvent event)
        {
            switch(event.getAction())
            {
            case MotionEvent.ACTION_DOWN:
                 preX = (int)event.getRawX();
                 preY = (int)event.getRawY();
                 isMove = false;
                 if(params.width == icon_width_side_temp)
                     handler.sendMessage(handler.obtainMessage(
                                        MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3, 0));
                 break;
            case MotionEvent.ACTION_UP:              
                 if(isMoveToEdge == true)
                 {
                     if(params.width == icon_width_side_temp)
                         handler.sendMessage(handler.obtainMessage(
                                        MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3, 0));
                     handler.sendMessage(handler.obtainMessage(
                                                 MSG_FLOAT_VIEW_MOVE_TO_EDGE,this));                     
                 }
                 break;
            case MotionEvent.ACTION_MOVE:
                 x = (int)event.getRawX();
                 y = (int)event.getRawY();               
                 if(Math.abs(x-preX)>1||Math.abs(y-preY)>1)
                 {                   
                  isMoveToEdge = true;
                 }
                 if(Math.abs(x-preX)>5||Math.abs(y-preY)>5)
                     isMove = true;
                 if(params.width == icon_width_side_temp)
                     handler.sendMessage(handler.obtainMessage(
                                       MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED, 3, 0));
                 manager.move(this, x-preX, y-preY);
                 preX = x;
                 preY = y;
                 break;
            }
            return super.onTouchEvent(event);
        }
    }

通过layout文件生成一个FloatIconView,在onTouchEvent函数中当按下的时候,发送消息更新悬浮view,抬起即up事件时先更新悬浮view,然后再显示吸附到边上的动画。 当move的时候,判断每次位移至少5和像素则更新view位置,这样不断move不断更新就会形成连续的画面。

另一个FloatRecordView(录音的悬浮窗)道理相同,这里就不贴代码了,有兴趣可以下载demo自己编译跑一下。

在FloatIconView中定义一个handler,用于接收消息处理悬浮窗更新位置和吸附的动画

private void initHandler(){
           handler = new Handler(){
              @Override
              public void handleMessage(Message msg) 
              {
                    switch (msg.what) 
                    {
                    case MSG_REFRESH_VOLUME:
                        if(floatRecordView != null)
                            floatRecordView.updateVolume((int)msg.arg1);
                        break;
                    case MSG_FLOAT_VIEW_MOVE_TO_EDGE:
                        //更新悬浮窗位置的动画
                        moveAnimation((View)msg.obj);
                        break;
                    case MSG_REMOVE_FLOAT_VIEW:
                        if(msg.arg1 == 1)
                        {//此时已有floatview是floatIconView
                            if(floatIconView != null)
                            {//先移除一个floatview
                                winManager.removeView(floatIconView);
                                floatIconView = null;
                                floatRecordView = getFloatRecordView();
                                if(floatRecordView != null)
                                {   
                                   if(floatRecordView.getParent() == null)
                                   {//再加入一个新的floatview
                                      winManager.addView(floatRecordView, params);
                                      floatViewType = FLOAT_RECORD_VIEW_TYPE;
                                   }
                                   if(mHandler != null)
                                   {
                                     mHandler.sendMessage(mHandler.obtainMessage(
                                             MessageConst.CLIENT_ACTION_START_CAPTURE));
                                     IS_RECORD_FROM_FLOAT_VIEW_IDLE = true;
                                   }
                                }
                            }
                        }
                        else
                        {//此时已有floatview是floatRecordView即录音的floatview
                           if(floatRecordView != null)
                           {//先移除一个floatview
                               winManager.removeView(floatRecordView);
                               floatRecordView = null;
                           }
                           floatIconView = getFloatIconView();
                           if(floatIconView != null)
                           {
                              if(floatIconView.getParent() == null)
                              {/再加入一个新的floatview
                                  winManager.addView(floatIconView, params);
                                  floatViewType = FLOAT_ICON_VIEW_TYPE;
                                  setViewOnClickListener(floatIconView);
                              }
                              //可能需要有吸附动画
                              moveAnimation(floatIconView);
                           }
                        }
                        break;
                    case MSG_UPDATE_VIEW_SENDING_TO_SERVER:
                        if(floatRecordView != null)
                        {
                            floatRecordView.updateSendingToServerView();
                            floatRecordView.setTitle("努力识别中");
                        }
                        break;
                    case MSG_UPDATE_ROTATE_VIEW:
                        if(floatRecordView != null)
                        {
                            floatRecordView.rotateview.startRotate();   
                        }
                        break;                  
                    case MSG_UPDATE_FLOAT_VIEW_AFTER_CHANGED:
                        //1,2是吸附到左边还是右边,3是拖动到中间显示放大的悬浮窗icon
                        if(msg.arg1 == 1)
                            changeFloatIconToSide(false);
                        else if(msg.arg1 == 2)
                            changeFloatIconToSide(true);
                        else if(msg.arg1 == 3)
                            changeFloatIconToNormal();              
                        break;
                    case MSG_UPDATE_FLOAT_VIEW_ON_SIDE:
                        if(msg.arg1 == 1)
                            updateFloatIconOnSide(true);
                        else if(msg.arg1 == 2)
                            updateFloatIconOnSide(false);
                        break;
                    case MSG_START_ACTIVITY:
                        hide();
                        Intent intent = new Intent(mContext,MusicActivity.class);
                        intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
                        intent.putExtra(START_FROM_FLOAT_VIEW, true);
                        IS_START_FROM_FLOAT_VIEW_IDLE = true;
                        mContext.startActivity(intent);
                        break;  
                    }
              }
           };
     }

那么,怎样做到点击吸附屏幕边缘的悬浮按钮,切换成录音的悬浮窗呢?

public void show()
{
        isHide = false;

        floatIconView = getFloatIconView();
        if(floatIconView != null)
        {
             if(floatIconView.getParent() == null)
             {
                  winManager.addView(floatIconView, params);
                  floatViewType = FLOAT_ICON_VIEW_TYPE;
             }
             if(floatRecordView != null)
             {
                 handler.sendMessage(handler.obtainMessage(
                                                       MSG_REMOVE_FLOAT_VIEW, 2, 0));       
             }
             floatIconView.setOnClickListener(new OnClickListener(){
                @Override
                public void onClick(View v) {
                    if(floatIconView.isMove || floatIconView.isMoveToEdge)
                    {
                        floatIconView.isMove = false;
                        return;
                    }
                    winManager.removeView(floatIconView);
                    floatIconView = null;
                    floatRecordView = getFloatRecordView();
                    if(floatRecordView != null)
                    {
                        if(floatRecordView.getParent() == null)
                        {
                            winManager.addView(floatRecordView, params);
                            floatViewType = FLOAT_RECORD_VIEW_TYPE;
                        }
                        if(mHandler != null)
                        {
                          mHandler.sendMessage(mHandler.obtainMessage(
                                         MessageConst.CLIENT_ACTION_START_CAPTURE));
                          IS_RECORD_FROM_FLOAT_VIEW_IDLE = true;
                        }
                    }

                }                
             });
        }
}

在show函数中,设置了floatIconView的点击事件,移除小的悬浮吸附按钮,加入录音的悬浮窗view并启动录音。

2.FloatViewIdleService

为什么要定义这个service? 这个service用途是,定时扫描是否在待机桌面,如果是待机桌面则显示floatview,否则隐藏。

public class FloatViewIdleService extends Service {

    private static Handler mHandler;  
    private FloatViewIdle floatViewIdle;
    private final static int REFRESH_FLOAT_VIEW = 1;
    private boolean is_vertical = true;
    @Override
    public void onCreate() {
        super.onCreate();
        initHandler();      
    }

    @Override
    public int onStartCommand(Intent intent, int flags, int startId) {
        mHandler.sendMessageDelayed(mHandler.obtainMessage(REFRESH_FLOAT_VIEW), 500);
        FloatViewIdle.IS_START_FROM_FLOAT_VIEW_IDLE = false;
        is_vertical = true;
        return START_STICKY;
    }
    protected void initHandler() {
        mHandler = new Handler() {
            @Override
            public void handleMessage(Message msg) {
                switch (msg.what) {
                case REFRESH_FLOAT_VIEW://1s发送一次更新floatview消息
                    updateFloatView();
                    mHandler.sendMessageDelayed(
                               mHandler.obtainMessage(REFRESH_FLOAT_VIEW), 1000);
                    break;
                }
            }
        };
    }

     private void updateFloatView()
     {
        boolean isOnIdle = isHome();//判断是否在待机界面
        floatViewIdle = FloatViewIdle.getInstance(FloatViewIdleService.this);       
        if(isOnIdle)
        { //待机界面则显示floatview            
            if(floatViewIdle.getFloatViewType() == 0)
            {               
                floatViewIdle.show();
            }
            else if(floatViewIdle.getFloatViewType() == 
                                        floatViewIdle.FLOAT_ICON_VIEW_TYPE||  
                    floatViewIdle.getFloatViewType() == 
                                        floatViewIdle.FLOAT_RECORD_VIEW_TYPE)
            {
                if(this.getResources().getConfiguration().orientation ==
                                          Configuration.ORIENTATION_LANDSCAPE) 
                {
                    if(is_vertical == true)
                    {
                       floatViewIdle.swapWidthAndHeight();  
                       is_vertical = false;
                    }
                }
                else if(this.getResources().getConfiguration().orientation == 
                                               Configuration.ORIENTATION_PORTRAIT)
                {
                    if(is_vertical == false)
                    {
                       floatViewIdle.swapWidthAndHeight();
                       is_vertical = true;
                    }
                }   
            }
        }
        else
        {//否则隐藏floatview
            floatViewIdle.hide();               
        }
     }

     private boolean isHome() 
     {  
        ActivityManager mActivityManager = (ActivityManager) 
                                       getSystemService(Context.ACTIVITY_SERVICE);  
        List<RunningTaskInfo> rti = mActivityManager.getRunningTasks(1); 
        try{ 
        if(rti.size() == 0)
        {
            return true;
        }else
        {
            if(rti.get(0).topActivity.getPackageName().
                                               equals("com.olami.floatviewdemo"))
                return false;
            else
                return getHomes().contains(rti.get(0).topActivity.getPackageName()); 
            }
        }
        catch(Exception e)
        {       
           return true;
        }
     }  

     private List<String> getHomes() 
     {  
        List<String> names = new ArrayList<String>();  
        PackageManager packageManager = this.getPackageManager();  
        Intent intent = new Intent(Intent.ACTION_MAIN);  
        intent.addCategory(Intent.CATEGORY_HOME);  
        List<ResolveInfo> resolveInfo = packageManager.queryIntentActivities(intent,  
                PackageManager.MATCH_DEFAULT_ONLY);  
        for (ResolveInfo ri : resolveInfo) {  
            names.add(ri.activityInfo.packageName);  
        }  
        return names;  
     }  

    @Override
    public void onDestroy() {
        super.onDestroy();
        if(floatViewIdle != null)
           floatViewIdle.setFloatViewType(0);
    }
    @Override
    public IBinder onBind(Intent intent) {
        return null;
    }

}

3.启动语音识别

在另一个VoiceSdkService(另一个处理录音服务业务的service)中,当接收到悬浮窗按钮点击事件消息时,则启动录音服务,录音结束后会在onResult回调中收到服务器返回的结果。

本例用的是olami语音识别,语义理解引擎,olami支持强大的用户自定义语义,能更好的解决语义理解。 比如同义理解的时候,我要听三国演义,我想听三国演义,听三国演义这本书,类似的说法有很多,olmai就可以为你解决这类的语义理解,olami语音识别引擎使用比较简单,只需要简单的初始化,然后设置好回调listener,在回调的时候处理服务器返回的json字符串即可,当然语义还是要用户自己定义的。

public void init()
{
    initHandler();
    mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);
    TelephonyManager telephonyManager=(TelephonyManager) this.getSystemService(
    (this.getBaseContext().TELEPHONY_SERVICE);
    String imei=telephonyManager.getDeviceId();
    mOlamiVoiceRecognizer.init(imei);//设置身份标识,可以填null

    mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);//设置识别结果回调listener
    mOlamiVoiceRecognizer.setLocalization(
    OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);//设置支持的语音类型,优先选择中文简体
    mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1",
                            "asr","68bff251789b426896e70e888f919a6d","nli");  
    //注册Appkey,在olami官网注册应用后生成的appkey
    //注册api,请直接填写“asr”,标识语音识别类型
    //注册secret,在olami官网注册应用后生成的secret
    //注册seq ,请填写“nli”

    mOlamiVoiceRecognizer.setVADTailTimeout(2000);//录音时尾音结束时间,建议填//2000ms
    //设置经纬度信息,不愿上传位置信息,可以填0 
    mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009); 

在VoiceSdkService中定义OlamiVoiceRecognizerListener用于处理录音时的回调

onError(int errCode)//出错回调,可以对比官方文档错误码看是什么错误
onEndOfSpeech()//录音结束
onBeginningOfSpeech()//录音开始
onResult(String result, int type)//result是识别结果JSON字符串
onCancel()//取消识别,不会再返回识别结果
onUpdateVolume(int volume)//录音时的音量,1-12个级别大小音量

本文用的是在线听书的例子,当收到服务器返回的消息是,进入如下函数: 在下面的函数中,通过解析服务器返回的json字符串,提取用户需要的语义理解字段进行处理

private void processServiceMessage(String message)
    {
        String input = null;
        String serverMessage = null;
        try{
            JSONObject jsonObject = new JSONObject(message);
            JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");
            JSONObject jObj = jArrayNli.optJSONObject(0);
            JSONArray jArraySemantic = null;
            if(message.contains("semantic"))
              jArraySemantic = jObj.getJSONArray("semantic");
            else{
                input = jsonObject.optJSONObject("data").optJSONObject("asr").
                optString("result");
                sendMessageToActivity(MessageConst.
                                     CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);
                serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();
                sendMessageToActivity(MessageConst.
                        CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0, 0, null, serverMessage);
                return;
            }
            JSONObject jObjSemantic;
            JSONArray jArraySlots;
            JSONArray jArrayModifier;
            String type = null;
            String songName = null;
            String singer = null;


            if(jObj != null) {
                type = jObj.optString("type");
                if("musiccontrol".equals(type))
                {
                    jObjSemantic = jArraySemantic.optJSONObject(0);
                    input = jObjSemantic.optString("input");
                    jArraySlots = jObjSemantic.optJSONArray("slots");
                    jArrayModifier = jObjSemantic.optJSONArray("modifier");
                    String modifier = (String)jArrayModifier.opt(0);
                    if((jArrayModifier != null) && ("play".equals(modifier)))
                    {
                        if(jArraySlots != null)
                           for(int i=0,k=jArraySlots.length(); i<k; i++)
                           {
                               JSONObject obj = jArraySlots.getJSONObject(i);
                               String name = obj.optString("name");
                               if("singer".equals(name))
                                   singer = obj.optString("value");
                               else if("songname".equals(name))
                                   songName = obj.optString("value");

                           }
                    }else if((modifier != null) && ("stop".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            if(mBookUtil.isPlaying())
                                mBookUtil.stop();
                    }else if((modifier != null) && ("pause".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            if(mBookUtil.isPlaying())
                                mBookUtil.pause();
                    }else if((modifier != null) && ("resume_play".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            mBookUtil.resumePlay();
                    }else if((modifier != null) && ("add_volume".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            mBookUtil.addVolume();
                    }else if((modifier != null) && ("del_volume".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            mBookUtil.delVolume();
                    }else if((modifier != null) && ("next".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            mBookUtil.next();
                    }else if((modifier != null) && ("previous".equals(modifier)))
                    {
                        if(mBookUtil != null)
                            mBookUtil.prev();
                    }else if((modifier != null) && ("play_index".equals(modifier)))
                    {
                        int position = 0;
                        if(jArraySlots != null)
                               for(int i=0,k=jArraySlots.length(); i<k; i++)
                               {
                                   JSONObject obj = jArraySlots.getJSONObject(i);
                                   JSONObject jNumDetial = obj.getJSONObject("num_detail");
                                   String index = jNumDetial.optString("recommend_value");
                                   position = Integer.parseInt(index) - 1;
                               }
                        if(mBookUtil != null)
                            mBookUtil.skipTo(position);
                    }
                }
            }
            if(songName != null)
            {
                if(singer != null)
                {

                }else{
                    mBookUtil.searchBookAndPlay(songName,0,0);
                }
            }else if(singer != null)
            {
                mBookUtil.searchBookAndPlay(songName,0,0);
            }
            serverMessage = jObj.optJSONObject("desc_obj").opt("result").toString();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
        //发送消息更新语音识别的文字
        sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);
        //发送消息更新服务器返回的结果字符串
        sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 
                                                    0, 0, null, serverMessage);

    }

以我要听三国演义这句语音,服务器返回的数据如下:

{
    "data": {
        "asr": {
            "result": "我要听三国演义",
            "speech_status": 0,
            "final": true,
            "status": 0
        },
        "nli": [
            {
                "desc_obj": {
                    "result": "正在努力搜索中,请稍等",
                    "status": 0
                },
                "semantic": [
                    {
                        "app": "musiccontrol",
                        "input": "我要听三国演义",
                        "slots": [
                            {
                                "name": "songname",
                                "value": "三国演义"
                            }
                        ],
                        "modifier": [
                            "play"
                        ],
                        "customer": "58df512384ae11f0bb7b487e"
                    }
                ],
                "type": "musiccontrol"
            }
        ]
    },
    "status": "ok"
}

1)解析出nli中type类型是musiccontrol,这是语法返回app的类型,而这个在线听书的demo只关心musiccontrol这 个app类型,其他的忽略。

2)用户说的话转成文字是在asr中的result中获取
3)在nli中的semantic中,input值是用户说的话,同asr中的result。
modifier代表返回的行为动作,此处可以看到是play就是要求播放,slots中的数据表示歌曲名称是三国演义。
那么动作是play,内容是歌曲名称是三国演义,在这个demo中调用
mBookUtil.searchBookAndPlay(songName,0,0);会先查询,查询到结果会再发播放消息要求播放,我要听三国演义这个流程就走完了。

关于在线听书请看博文:blog.csdn.net/ls0609/arti…

4.源码下载链接

pan.baidu.com/s/1o8OELdC

5.相关链接

语音记账demo:blog.csdn.net/ls0609/arti…

olami开放平台语法编写简介:blog.csdn.net/ls0609/arti…

olami开放平台语法官方介绍:cn.olami.ai/wiki/?mp=nl…