WO2016005989A1 - Method and apparatus for wireless operation of mobile computing device - Google Patents

Abstract

The present invention provide a method of wirelessly connecting mobile computing device to external peripherals, including mobile computing device capable of wirelessly connecting to peripherals and capable of sending wirelessly representation of its display to peripherals connecting apparatus.

Description

METHOD AND APPARATUS FOR WIRELESS OPERATION OF MOBILE COMPUTING DEVICE

Field of the Invention

[1] This invention generally pertains to computer peripheral systems and more specifically to wireless connection of computer peripherals to mobile computing device.

Background of the invention

[2] Recently one can witness revolution in the way we operate personal computing devices. While originally personal computing devices were stationary, nowadays more mobile personal computing devices are purchased and used than stationary ones. Such mobile personal computing devices include laptops, tablets, smart-phones and wearable devices like smart- watches and smart-glasses. Device mobility allows users to have their computing device always within reach, increasing frequency and effectiveness of person-machine interaction. Moreover such close and frequent interaction calls for increased personalization of such mobile personal computing device.

[3] Usage of diverse mobile personal computing devices are encouraged in workplace by Bring Your Own Device (BYOD) policies.

[4] A common problem with device mobility is that, mobile devices have to be small, light and easy to carry around. Historically, being light and small required significant sacrifice in computing power of mobile device, but current trends in miniaturization and power efficiency make this assumption obsolete. Unfortunately being small still causes significant degradation in usability. Small screen, small speakers and small input controls (touch surface, keys etc.) significantly limit efficiency of information exchange between user and the mobile computing device.

[5] Common solution to usability sacrifice is using multiple computing devices with different form factors. One will carry a smart-phone, which is mobile and pocketable, while using additional desktop computer when smart-phone small I/O devices cannot support adequate productivity. Unfortunately using more than one computing device creates discontinuity and significantly sacrifices personalization and fluidity of person-machine interaction.

[6] Common solution to the problem above is dynamic adaptation of personal computing device to required form factor by connecting external I/O peripherals. A common example of such solution may be seen in connecting of laptop computers to external display, keyboard and mouse by means of docking station. Unfortunately, connecting peripherals as described above requires mechanical connection and disconnection procedure limiting user's ability to perform fast switch of form factor, for example moving from his desk to meeting room. Moreover, many mobile devices lack physical/hardware connectivity and software support to connect external peripherals. In case of wearable devices physical connection to peripherals is not a reasonable option.

[7] Common solution to the above is usage of wireless connectivity to connect to peripherals.

Bluetooth is a common wireless technology to connect human input devices like, keyboard and mouse and audio devices like, speakers and headphones. Wireless connectivity of displays is possible with technologies like Miracast, WiDi or AirPlay Mirroring. Unfortunately connecting peripherals using wireless technologies is usually cumbersome manual process inhibiting fast switching of form factors.

[8] Another problem with common mobile personal computing devices is power supply. Usually batteries of such devices are not designed for whole day constant full duty operation and require connection to external power source in order to be used as the main personal computing device. As described above, connecting and disconnecting power supply mechanically is limiting user's ability to perform fast switch of form factor.

[9] Available solution to the problem above is wireless charging. Qi is a known standard for wireless inductive power transfer.

[10] Driving a motor vehicle is another modality with constrained support for using personal mobile computing device. Common in-built car infotainment systems may be considered as additional personal computing device creating discontinuity in user's communication with personal mobile computing device. Usually such infotainment systems are closed and allow limited possibilities to integrate user's personal computing device into driving experience.

[11] In view of the above, there is a need for a system and method capable of using single personal computing device in multiple form factors allowing fast switching of form factors to support user's whole day routine.

Summary of the invention

[12] Generally described the embodiments of the present invention provide a method to provide multiple usage form factors to mobile computing device, overcoming the problems disclosed in the prior art by leveraging wireless connectivity.

[13] The representative embodiments include a smart-phone capable of wirelessly connecting to keyboard, mouse and headset, capable of sending wirelessly representation of its display and capable of receiving energy using inductive power transfer. Another component of representative embodiments is peripherals connecting apparatus (PCA) connected to computer display, keyboard, mouse, audio-headset and power adapter by means of DVI/HDMI, USB, 3.5mm phone connectors and micro-USB connector respectively. Said physical device plays a role of wireless keyboard, wireless mouse, wireless headset, wireless display and wireless charger in respect to the said smart-phone allowing the user of the peripherals connected to said physical device to operate said smart-phone wirelessly.

[14] It should be noted that while the present invention describes a smart-phone as the mobile computing device, it should be obvious to the skilled in art that the smart-phone may be substituted by any mobile computing device (MCD), such as a tablet, laptop, smart-watch, smart-glass or any conjunction of devices allowing wireless connection of display, headset, mouse, keyboard and other peripherals.

[15] It should be noted that while the present invention describes a DVI/HDMI as the display interface, it should be obvious to the skilled in art that other display interfaces may be used, such as VGA, DisplayPort, DisplayLink, etc. [16] It should be noted that while the present invention describes a USB as the keyboard and mouse interface, it should be obvious to the skilled in art that other human interface device interfaces may be used, such as PS/2, Serial, IR, RF, etc.

[17] It should be noted that while the present invention describes a 3.5mm phone connector as the headset interface, it should be obvious to the skilled in art that other audio device interfaces may be used, such as USB Audio, RF, etc.

[18] According to the teachings of the present invention, a peripherals connecting apparatus (PCA) is provided for wireless connecting of wired computer peripherals to a mobile computing device (MCD) supporting wireless connection of peripherals. The apparatus includes keyboard controller, mouse controller, audio codec, display controller, processor and wireless communication device for communicating with the smart-phone. Preferably, power is provided to the smart-phone by an apparatus including inductive power transfer coil.

[19] Keyboard controller enables receipt of human interface device (HID) reports from the attached keyboard and provides these HID reports to a processor. The processor encodes the HID reports into suitable wireless format and enables the wireless communication device to send these HID reports to the smart-phone over wireless network link. Mouse controller enables receipt of human interface device (HID) reports from the attached mouse and provide these HID reports to the processor. The processor encodes the HID reports into suitable wireless format and enables the wireless communication device to send these HID reports to the smart- phone over wireless network link. Wireless communication device enables receipt of encoded smart-phone display frames and provide them to the processor. The processor decodes the display frames and provides them to display controller, which enables display of these frames on the attached display. Wireless communication device enables receipt of encoded digital audio stream from smart-phone and provide it to the processor. The processor decodes the audio stream and provides it to DAC device which enables playing of this stream on the attached speaker/headphones. ADC device enables conversion of audio from microphone input into digital form and provide digital audio stream to the processor. The processor encodes the audio stream into suitable wireless format and enables the wireless communication device to send this audio stream to the smart-phone over wireless network link.

[20] According to one aspect of the invention a method is provided for switching a modality of mobile computing device, the method including the steps of:

a. Connecting mouse, keyboard, headset and display to peripherals connecting apparatus. b. Advertising the peripherals connecting apparatus to the mobile computing device as wireless keyboard, wireless mouse, wireless headset, and wireless display.

c. Upon detection of a trigger to switch modality, the modality controller of the mobile computing device will instruct the mobile computing device to connect to said identified wireless keyboard, wireless mouse, wireless headset and wireless display

d. Peripherals connecting apparatus emulating wireless keyboard, wireless mouse, wireless headset and wireless display by responding to said mobile computing device connection attempt according to the protocol of respectively wireless keyboard, wireless mouse, wireless headset and wireless display.

e. Upon completion of establishment of wireless connection with mobile computing device, peripherals connecting apparatus will continue emulating wireless keyboard, wireless mouse, wireless headset and wireless display by tunneling the data from the peripherals connecting the apparatus to mobile computing device and by tunneling the data from mobile computing device to the peripherals connected to the apparatus.

[21] Optionally the trigger to switch modality may be selected from the group of proximity, quality of wireless signal, availability of wireless power transmission, limited range wireless communication.

[22] According to another aspect of the invention a method is provided for fast switching a modality of mobile computing device, the method including the steps of:

a. Connecting mouse, keyboard, headset and display to peripherals connecting apparatus. b. Advertising the peripherals connecting apparatus to the mobile computing device as wireless keyboard, wireless mouse, wireless headset, and wireless display. c. Upon detection of a trigger to potential switch of modality, the modality controller of the mobile computing device will instruct the mobile computing device to connect to said identified wireless keyboard, wireless mouse, wireless headset and wireless display d. Peripherals connecting apparatus emulating wireless keyboard, wireless mouse, wireless headset and wireless display by responding to said mobile computing device connection attempt according to the protocol of respectively wireless keyboard, wireless mouse, wireless headset and wireless display.

e. Peripherals connecting apparatus delaying completion of the said mobile computing device connection attempt by delaying responding to the last message of the protocol for the respective wireless peripheral connection.

f. Upon detection of a trigger to effective switch of modality, peripherals connecting apparatus will complete said wireless connection by responding to the delayed messages.

g. Upon establishing of wireless connection with mobile computing device, peripherals connecting apparatus continue emulating wireless keyboard, wireless mouse, wireless headset and wireless display by tunneling the data from the peripherals connecting the apparatus to mobile computing device and by tunneling the data from mobile computing device to the peripherals connected to the apparatus.

[23] Optionally the trigger to potential switch of modality may be selected from the group of proximity, quality of wireless signal, availability of wireless power transmission, limited range wireless communication, location information

[24] Optionally the trigger to effective switch of modality may be selected from the group of proximity, quality of wireless signal, availability of wireless power transmission, limited range wireless communication.

[25] According to another aspect of the invention a method is provided for minimizing delay between mouse movement and cursor movement on the screen of mobile computing device, the method including the steps of:

a. Connecting mouse and display to peripherals connecting apparatus (PCA). b. Wirelessly connecting peripherals connecting apparatus with mobile computing device (MCD) by said peripherals connecting apparatus acting in accordance with the protocol of wireless mouse and wireless display

c. Emulating wireless mouse by tunneling the data from the mouse connecting the PCA to MCD using wireless mouse protocol.

d. Estimating the relation between the movement of mouse connected to PCA and cursor movement on the screen of MCD

e. Upon detection of a mouse movement event arriving from said connected mouse, PCA performs the following steps

i. Tunneling the mouse movement event to MCD over wireless connection i^'i. Estimating the expected new location of the cursor on the screen of MCD according to said mouse movement and said relation estimated in step (d). iii. Presenting cursor at said estimated location on the display connected to PCA.

[26] Alternative representative embodiments of the present invention includes vehicle infotainment system (VIS) connected to PCA acting as display connected to PCA in the system, apparatus and methods described above. PCA is facilitated to transfer display and audio output to said VIS to be presented by said VIS display and speakers.

[27] According to another aspect of the invention a method is provided transferring display output of PCA to VIS using VIS storage device interface in accordance with the teachings of US8606071B2, the method including the steps of:

1. PCA emulating storage device connected to VIS using storage device interface

2. PCA receiving the display output stream of MCD over the wireless link

3. PCA converting the display output stream into AV format playable by VIS by means of storage device interface

4. PCA facilitates VIS to consume and present the converted AV stream on VIS display

[28] Optionally the conversion of display output stream can include any combination of copying, trans-coding and transport format adjustment [29] According to another aspect of the invention a method is provided transferring display output of PCA to VIS using MirrorLink protocol, the method including the steps of:

1. Connecting PCA to VIS using MirrorLink protocol

2. PCA receiving the display output stream of MCD over the wireless link

3. PCA converting the display output stream into MirroLink compatible format displayable by IS

4. PCA sending the converted display output stream to VIS over MirrorLink.

[30] Optionally the conversion of display output stream can include any combination of copying, decoding and transport format adjustment

[31] An embodiment is an example or implementation of the inventions. The various appearances of "one embodiment," "an embodiment" or "some embodiments" do not necessarily all refer to the same embodiments. Although various features of the invention may be described in the context of a single embodiment, the features may also be provided separately or in any suitable combination. Conversely, although the invention may be described herein in the context of separate embodiments for clarity, the invention may also be implemented in a single embodiment.

[32] Reference in the specification to "one embodiment", "an embodiment", "some embodiments" or "other embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiments, but not necessarily all embodiments, of the inventions. It is understood that the phraseology and terminology employed herein is not to be construed as limiting and are for descriptive purpose only.

Brief description of the drawings

[33] The present invention will become fully understood from the detailed description given herein below and the accompanying drawings, which are given by way of illustration and example only and thus not limitative of the present invention, and wherein: [34] Fig 1 schematically illustrates a peripherals connecting system containing mobile computing device connected to peripherals connecting apparatus.

[35] Fig 2 is an in depth description of the preferred embodiment system, describing main building blocks of peripherals connecting apparatus.

[36] Fig 3 is a detailed schematic illustration of method for MCD performing modality switch by wirelessly connecting external peripherals.

[37] Fig 4 is a detailed schematic illustration of method for MCD performing fast modality switch by wirelessly connecting external peripherals.

[38] Fig 5 is a detailed schematic illustration of method for minimizing delay between mouse movement and respective cursor movement on the display attached to peripherals connecting apparatus.

[39] Fig 6 is a detailed schematic illustration of system to display MCD's display output on vehicle infotainment device.

Detailed description of the preferred embodiment

[40] Fig 1 schematically illustrates a peripherals connecting system 100 containing MCD 110 connected to PCA 120.

[41] The system 100 is capable of presenting display output rendered by MCD 110 on the display device 101 attached to PCA 120. MCD 110 encodes display output frames and sends them over wireless network 105. PCA 120 receives display output of MCD 110 using wireless network 105 and presents it on the display device 101. It should be obvious to the skilled in art that any device allowing presentation of video information such as projector, head-up display or TV can be considered as display 101 by this invention.

[42] The system 100 is capable of playing audio rendered by MCD 110 on the headphones of the headset 104 attached to PCA 120. MCD 110 encodes audio and sends it over wireless network 105. PCA 120 receives audio rendered by MCD 110 using wireless network 105 and plays it on the headphones of the headset 104. It should be obvious to the skilled in art that any device which allows playing of audio such as speakers, headphones or even speakers embedded into display device 101 may be considered as headset 104 by this invention.

[43] The system 100 is capable of delivering to MCD110 audio signal from the microphone of the headset 104 attached to PCA 120. PCA 120 sends audio signal to MCD 110 using wireless network 105. MCD 110 receives audio using wireless network 105 and treats it as MCD's 110 audio input. It should be obvious to the skilled in art that any device providing audio signal input such as analog or digital microphone or analog or digital sound input may be considered as microphone of headset 104 by this invention.

[44] The system 100 is capable of delivering to MCD 110 the keyboard events of the keyboard 102 attached to PCA 120. PCA 120 sending keyboard events to MCD 110 using wireless network 105. MCD 110 receives keyboard events using wireless network 105 and treats it as MCD's 110 keyboard input. It should be obvious to the skilled in art that any device providing key-press events such as keypad, barcode scanner or gaming controller may be considered as keyboard 102 by this invention. [45] The system 100 is capable of delivering to MCD 110 the mouse events of the mouse 103 attached to PCA 120. PCA 120 sends mouse events to MCD 110 using wireless network 105. MCD 110 receives mouse events using wireless network 105 and treats it as MCD's 110 pointing input. It should be obvious to the skilled in art that any device providing pointing events such as trackball, trackpad, digitizer or touchscreen may be considered as mouse 103 by this invention.

[46] The system 100 is capable of delivering to MCD 110 power by means of inductive power transfer coil 106 attached to PCA 120. MCD 110 receives wireless power transfer using corresponding wireless power receiver and uses received power to power itself or charge it's batteries. It should be obvious to the skilled in art that any device allowing wireless transfer of power such as source of electromagnetic radiation, induction or conduction may be considered as power transfer coil 106 by this invention.

[47] In an in depth description of the preferred embodiment system depicted further in fig 2 describing main building blocks of PCA 120 and describing a best mode of carrying out the teachings described in paragraphs [18]-[19]

[48] Keyboard Controller 224 is responsible for communicating with attached keyboard.

Keyboard Controller 224 captures key press events from the keyboard attached to PCA and transfers them to Processor 220 connected to Keyboard Controller 224. Preferably the keyboard is attached using USB connection and using HID over USB protocol to send keyboard events to Keyboard Controller 224. Preferable Keyboard Controller, 224 will be implemented by USB host controller and by USB and HID driver software running on Processor 220 controlling such USB host controller. Upon receiving keyboard event the Processor 220 encodes it in format appropriate for wireless keyboard protocol understandable by MCD 110 and sends it to connected Wireless Network Card 236. Wireless Network Card 236 sends encoded keyboard event to MCD 110 over wireless link. Preferably MCD 110 uses Bluetooth technology to receive wireless keyboard events and Processor 220 uses HID over Bluetooth protocol, well known in art, as format for encoding keyboard events. It should be obvious to the skilled in art that any technology allowing MCD 110 to consume keyboard events over wireless link may be used for this aspect. [49] Mouse Controller 226 is responsible for communicating with attached mouse. Mouse Controller 226 captures pointing events from the mouse attached to PCA and transfers them to Processor 220 connected to Mouse Controller 226. Preferably the mouse is attached using USB connection and using HID over USB protocol to send pointing events to Mouse Controller 226. Preferable Mouse Controller 226 is implemented by USB host controller and by USB and HID driver software running on Processor 220 controlling such USB host controller. Upon receiving mouse event the Processor 220 encodes it in format appropriate for the wireless mouse protocol understandable by MCD 110 and sends it to connected Wireless Network Card 236. Wireless Network Card 236 sends encoded pointing event to MCD 110 over wireless link. Preferably MCD 110 uses Bluetooth technology to receive wireless mouse events and Processor 220 uses HID over Bluetooth protocol, well known in art, as format for encoding pointing events. It should be obvious to the skilled in art that any technology allowing MCD 110 to consume mouse events over wireless link may be used for this aspect. Some MCD, such as contemporary iOS devices, do not support natively consumption of mouse events over wireless link. Additional components may be required to support such scenario. One such solution may include software component embedded into applications executed by MCD. Such component communicates with PCA over Bluetooth, receives pointing events and facilitates the application to interpret them similarly to touch events delivered by MCD's operating system. In case of iOS, HID over GATT with Bluetooth 4.0 may be used by application component to communicate with PCA.

[50] Wireless Network Card 236 facilitates receipt of encoded display frames sent by MCD using wireless network. Processor 220 facilitates A/V Decoder 222 to decode said received encoded display frames. Processor 220 facilitates Display Controller 230 to present said decoded display frames on the display attached to PCA 120. Preferably MCD uses one of the standard protocols such as Miracast or AirPlay Mirroring to wirelessly send display output.

[51] Wireless Network Card 236 facilitates receipt of encoded audio sent by MCD using wireless network. Processor 220 facilitates A/V Decoder 222 to decode said received audio. Processor 220 facilitates Audio Codec 228 to play said decoded audio on the headphones attached to PCA 120. Preferably MCD uses one of the standard protocols such as Bluetooth A2DP to witlessly send audio to PCA. Alternatively audio may be sent interleaved with display output using protocols such as Miracast or AirPlay Mirroring mentioned above.

[52] Audio Codec 228 is facilitated to capture audio input from headset microphone attached to PCA. Processor 220 obtains such captured audio from Audio Codec 228. Processor 220 encodes said captured audio into format appropriate for the wireless audio input protocol understandable by MCD 110 and sends it to connected Wireless Network Card 236. Wireless Network Card 236 sends encoded audio to MCD 110 over wireless link. Preferably MCD 110 uses Bluetooth technology to receive audio and Processor 220 uses Bluetooth HFP/HSP protocol, well known in art, as format for encoding audio. It should be obvious to the skilled in art that any technology allowing MCD 110 to consume audio over wireless link may be used for this aspect.

[53] Wireless Power Module 238 facilitates inductive power transfer coil attached to PCA to transfer power to MCD 110 located in the range of wireless power transfer.

[54] Preferably Processor 220 executes software algorithms to achieve functionality described above. Such execution uses RAM 232 and Data Storage 234 by methods well known in art for software execution.

[55] The methods and modules described above may be implemented both in pure hardware system and in hybrid hardware and software system.

[56] Representative embodiment of the invention aspect described in paragraphs [20]-[21] depicted in Fig 3, a detailed schematic illustration of method 300 of MCD performing modality switch by wirelessly connecting external peripherals. During the setup phase (steps 302 and 304) all external peripherals (mouse, keyboard, headset, and display) are connected to PCA and PCA is powered up (step 302), then MCD is paired with PCA using appropriate wireless peripheral pairing protocols (step 304). In the preferred embodiment industry standard protocols for wireless peripherals supported by MCD are used. HID over Bluetooth is used for connecting keyboard and mouse, Bluetooth A2DP/HFP/HSP is used for connecting headset and Miracast/AirPlay Mirroring is used for connecting display. PCA advertises itself as wireless peer for such protocols and MCD standard procedure for pairing such wireless peer is followed by the MCD user. Optionally pairing process is assisted by software component deployed to MCD. For MCDs having limitation on ability of add-on software components to assist pairing, like iOS devices, alternative assistance method may be used. As soon as iOS device completes pairing with PCA using wireless keyboard protocol, PCA sends set of keyboard press events to MCD to automate MCD in process of pairing other peripheral protocols.

[57] During stand-by phase (step 310) PCA waits for trigger to switch modality. The trigger to switch modality may be deduced from proximity sensor embedded either into PCA or MCD, quality of wireless communication signal between PCA and MCD (for example Bluetooth RSSI), wireless communication round-trip time, short range wireless communication like NFC or RFID, mechanical communication like sound waves, availability of wireless power transmission, electric conductivity like electric switch, magnetic field measurement, human command or mix of the above. The appropriate measurements may be done by either PCA or MCD. In case the measurement is done by MCD it may be signaled to PCA by wireless communication. During the stand-by phase PCA avoids relaying communication between attached peripherals and MCD. Some of the methods to achieve it may be PCA advertising such wireless peripheral be off-line or out of range.

[58] Upon detection of the trigger to switch modality the system enters the connecting phase (steps 320 and 322). PCA makes all paired wireless peripherals available for connection by MCD and leverages Modality Controller 290 deployed to MCD to force MCD connection to said advertised wireless peripherals (step 320). For MCDs with limited possibilities for Modality Controller to force MCD to connect to wireless peripherals (iOS devices) PCA uses method described in paragraph [56] to automate MCD to achieve such task. PCA satisfies MCD's requests to connect to wireless peripherals according to the appropriate wireless protocols to facilitate connection of attached peripherals over wireless link (step 322). In some cases appropriate wireless peripheral protocol supports establishment of connection from the device side, PCA initiates such connection on behalf of such peripheral.

[59] Upon completion of connecting phase the method enters connected state (steps 324 to 372).

PCA relays data between peripherals and MCD and acts upon receiving data to relay (324) as described in paragraphs [48]-[55]: a. Relaying communication between CD and display by decoding received display frames (340) and presenting them on attached display (342).

b. Relaying communication between MCD and sound output device (headphones) by decoding received sound fragment (350) and playing it on sound output device (352).

c. Relaying communication between keyboard/mouse and MCD by converting received keyboard/mouse event to appropriate wireless format (360) and sending it to MCD over wireless link (362)

d. Relaying communication between headset microphone and MCD by encoding captured microphone audio into appropriate wireless format (370) and sending it to MCD over wireless link (372)

[60] Representative embodiment of the invention aspect described in in paragraphs [22]-[24] depicted in Fig 4, a detailed schematic illustration of method 400 of MCD performing fast modality switch by wirelessly connecting external peripherals. Method 400 is variation of method 300 with following differences.

[61] Establishment of wireless peripheral connection may be a lengthy procedure delaying completion of modality switch.

[62] Step 406 introduces an idea of speculative estimation of upcoming modality switch. PCA waits for trigger to start preparation for modality switch. Such trigger may by anything described in paragraph [57] having similar or different threshold values and characteristics. Upon detection of the said trigger PCA follows to step 320 as described above and follows to step 422.

[63] Step 422 is different from step 322 by the fact that PCA strives to prolong completion of establishment of wireless peripheral connection as long as it is possible according to the appropriate wireless protocol. PCA delays the last possible communication that concludes establishment of appropriate wireless connection, while waiting for trigger to complete the switch of modality (step 424). The trigger to complete switch of modality may be anything described in paragraph [57] usually having different threshold values and characteristics than the preparation trigger. Upon detection of said trigger (step 424), PCA completes establishment of wireless peripheral connection as soon as possible (step 426). PCA completes the last communication that was delayed in step 422.

The rest of method 400 is similar to method 300.

Representative embodiment of the invention aspect described in par [25] depicted in fig 5, a detailed schematic illustration of method 500 of minimizing delay between mouse movement and respective cursor movement on the display attached to PCA. Connecting peripherals over wireless link introduces significant delay in capturing user input by MCD and presenting output rendered by MCD. This may be noticeable in interactive scenarios, like moving mouse cursor.

Method 500 uses same setup, connecting and connected steps for wireless mouse as described in method 300 (steps 302, 304, 322, 324, 360 and 362) with addition of steps 560- 582.

Upon completion of relaying mouse event to MCD (362), PCA decides if it has enough knowledge to estimate the relationship between movement of the mouse that was just relayed and the expected movement of the mouse cursor on MCD screen that should occur as result of the mouse event processed by MCD (step 560).

If the answer to 560 is "Yes" then PCA estimates expected new location of the cursor on MCD screen. PCA uses the details of the mouse movement and the relation between the mouse movement and the cursor movement (570). Then PCA draws the cursor on the attached display according to new expected location (572).

If the answer to 560 is "No" then PCA waits until MCD cursor is moved by MCD to new location as a reaction to recent mouse movement. PCA acquires the new location of the MCD cursor (580). PCA may acquire new cursor location by software component hosted by MCD that receives new cursor location from MCD's operating system and sends it to PCA. Optionally PCA may acquire the new location of the cursor by analyzing the display frames that MCD sends to PCA to be displayed on the display attached to PCA. PCA uses the new location of the cursor to build/improve the model of relationship between mouse movements and MCD cursor movements (582). The model may be build using variety of machine learning techniques known in art. Quality of the prediction for new cursor location the model provides relatively to actual cursor location is considered next time the step 560 is evaluated.

[71] Representative embodiment of the invention aspect described in paragraphs [26]-[30] depicted in fig 6, a detailed schematic illustration of system 600 to display MCD's display output on vehicle infotainment device. It should be obvious to the skilled in art that any display device not capable to be connected using standard display connection but can be connected using method described below may be considered as a vehicle infotainment system display.

[72] System 600 is similar to system 100 with following difference. Instead of attaching display device 101 to PCA 120 as described in system 100, vehicle infotainment system 601 is connected to PCA 620 by means of USB connection (630 and 640).

[73] The system 600 is capable of presenting display output rendered by MCD 110 on the display of VIS 601 attached to PCA 620. MCD 110 encodes display output frames and sends them over wireless network 105. PCA 620 receives display output of MCD 110 using wireless network 105 and presents it on the display of VIS 601.

[74] One method to achieve the above is following the teaching of ti ¾606071 . The method includes the steps of:

a. PCA emulating storage device connected to VIS using storage device interface b. PCA receives the display output stream of MCD over the wireless link

c. PCA converts the display output stream into AV format playable by VIS by means of storage device interface

d. PCA facilitates VIS to consume and present the converted AV stream on VIS display

[75] Optionally the conversion of display output stream may include some combination of copying, trans-coding and transport format adjustment

[76] Another method is provided transferring display output of PCA to VIS using MirrorLink protocol. The method including the steps of:

a. Connecting PCA to VIS using MirrorLink protocol over USB

b. PCA receiving the display output frames of MCD over the wireless link as described in paragraph [50] c. PCA converting the display output frames into MirroLink compatible format displayable by VIS, following this method:

i. Decoding and rendering the received frames into PCA's frame-buffer 231 which is part of Display Controller 230

ii. Encoding the rendered frames using MirraLink FB format

d. PCA sends the converted display frames to VIS over MirrorLink.

e. PCA receives VIS keyboard/mouse events over MirrorLink protocol

f. PCA converts the keyboard/mouse events into appropriate wireless protocol.

g. PCA sends converted keyboard/mouse events to MCD over wireless link.

[77] Optionally the conversion of display output stream may include some of copying, decoding and transport format adjustment

[78] Optionally connecting between PCA and VIS using MirrorLink protocol over WiFi.

[79] Although the present invention has been described with reference to the preferred

embodiment and examples thereof, it will be understood that the invention is not limited to the details thereof. Various substitutions and modifications have been suggested in the foregoing description, and others will occur to those of ordinary skill in the art. Therefore, all such substitutions and modifications are intended to be embraced within the scope of the invention as defined in the following claims.

Claims

Claims

1. A method of wirelessly connecting mobile computing device (MCD) to external peripherals comprising:

a. MCD capable of wirelessly connecting to peripherals capable of sending wirelessly representation of its display, and

b. Peripherals connecting apparatus (PCA) connected to computer display, pointing device like mouse, keyboard and maybe connected to audio/visual apparatus, printing apparatus and power adapter by any known means of connection comprising: keyboard controller, mouse controller, audio codec, display controller, processor and wireless communication device for communicating with MCD.

c. Modality controller (MC) capable of instructing MCD to connect/disconnect/use/not use wireless peripherals and wireless display device.

Wherein keyboard controller receives human interface device (HID), HID reports from attached keyboard providing HID reports to a processor, and

Wherein processor encodes human interface device (HID) reports into wireless keyboard format enabling wireless communication device to send HID reports to MCD over wireless network link, and

Wherein mouse controller receives HID reports from attached pointing device providing HID reports to processor, and

Wherein processor encodes HID reports into suitable wireless pointing device format enabling wireless communication device to send HID reports to MCD over wireless network link, and

Wherein wireless communication device receives from MCD encoded MCD display frames and provides them to processor, and

Wherein processor decodes display frames and provides them to display controller, displaying frames on attached display, and

Wherein wireless communication device receives encoded digital audio stream from MCD and provide it to processor, and Wherein processor decodes the audio stream and provides it to DAC device for playing on attached audio output devices, and

wherein ADC device converts audio input from audio input into digital form and provide digital audio stream to processor, and

Wherein processor encodes audio stream into suitable wireless audio input format to be sent by wireless communication device to MCD over wireless network link.

2. A method as claimed in claim 1 for switching a modality of MCD comprising the steps of: a. Connecting peripherals like mouse, keyboard, display, printing and audio devices to PCA, and

b. Identifying PCA to MCD as respective wireless peripherals, and

c. Upon detecting a trigger to switch modality, MC instructs MCD to connect and use wireless peripherals identified by PCA, emulating wireless peripherals by responding to MCD connection attempt according to protocol of respective wireless peripherals, and

d. Upon establishing wireless connection with MCD according to protocol of respective wireless peripherals, PCA continues emulating wireless peripherals by tunneling data from peripherals to MCD and by tunneling the data from MCD to peripherals connected to apparatus.

3. A method as claimed in claim 2 wherein MC is deployed to MCD as either software or hardware component.

4. A method as claimed in claim 2 wherein MC instructs MCD to connect and use wireless peripherals identified by PCA by connecting PCA to MCD using wireless keyboard protocol, wherein PCA sends set of keyboard press events to MCD to automate MCD in process of pairing other peripheral protocols.

5. The method of claim 2 wherein the trigger to switch modality is selected from a group of proximity data, quality of wireless signal, availability of wireless power transmission, limited range wireless communication signal and physical location information.

6. A method as claimed in any one of claims 2-5 for fast switching a modality of MCD, wherein upon detecting the starting trigger to switch modality, MC instructs MCD to connect and switch to wireless peripherals identified by PCA, emulating wireless peripherals by responding to MCD connection attempt according to protocol of respective wireless peripherals, and

wherein PCA delays responding to at least one of the messages of said MCD connection attempt, PCA responds to the delayed message when the completing trigger is detected.

7. The method of claim 6 wherein the starting trigger and the completing trigger are selected from a group of proximity data, quality of wireless signal, availability of wireless power transmission, limited range wireless communication signal and physical location information.

8. A method as claimed in any one of claims 1-7 for minimizing delay between mouse movement and cursor movement on screen connected to PCA, including the steps of: a. Connecting mouse and display to PCA, and

b. Wirelessly connecting PCA to MCD acting in accordance with the protocol of wireless mouse and display, and

c. Emulating wireless mouse by tunneling the data to MCD using wireless mouse protocol, and

d. Estimating the relation between the movement of mouse connected to PCA and cursor position on the screen of MCD, and

e. Upon detection of mouse movement, PCA performs following steps:

i. Estimating the expected new position of cursor on screen of MCD according to mouse movement and estimated relation in step (d), and

ii. Presenting cursor at said estimated position on display connected to PCA, and iii. Tunneling mouse movement event to MCD over wireless connection f. Upon detection of new cursor position on MCD screen adjusting the estimation of step (d)

9. A method as claimed in claim 8 for minimizing delay between mouse movement and cursor movement on screen connected to PCA wherein PCA acquires new cursor position on MCD screen by software component hosted by MCD receiving new cursor position from MCD's operating system and sends it to PCA.

10. A method as claimed in claim 8 for minimizing delay between mouse movement and cursor movement on screen connected to PCA wherein PCA acquires the new position of the cursor on MCD screen by analyzing the display frames that MCD sends to PCA to be displayed on the display attached to PCA.

11. A method for wirelessly connecting MCD to vehicle infotainment system (VIS) screen and speakers comprising,

a. vehicle infotainment system (VIS) capable of presenting AV stream received from storage device interface, and

b. PCA capable of transferring audio/visual output to VIS to be displayed on VIS's display/speakers, and

c. MCD capable of wirelessly connecting to PCA and wirelessly sending display output to PCA.

Wherein display output of PCA is transferred to VIS using VIS storage device interface, including the steps of:

a. MCD connected to PCA over wireless link, and

b. PCA emulating storage device connected to VIS using storage device interface, and c. PCA receiving display output stream of MCD over wireless link, and

d. PCA converting display output stream into AV format playable by VIS by means of storage device interface, and

e. PCA facilitates VIS to consume and present the converted AV stream on VIS display and speakers

12. The method of claim 11 wherein conversion of display output stream includes any combination of copying, trans-coding and transport format adjustment

13. The method of claim 11 wherein transferring display output of PCA to VIS is performed by using MirrorLink protocol, including the steps of:

a. MCD connected to PCA over wireless link, and

b. Connecting PCA to VIS using MirrorLink protocol, and

c. PCA receives display output stream of MCD over wireless link, and d. PCA converts display output stream into MirroLink compatible format displayable by VIS, and

e. PCA sends converted display output stream to VIS over MirrorLink.

14. The method of claim 13 wherein conversion of display output stream includes any combination of copying, decoding and transport format adjustment