CN104104698A - Web data cache processing method, device and system - Google Patents
Web data cache processing method, device and system Download PDFInfo
- Publication number
- CN104104698A CN104104698A CN201310115413.XA CN201310115413A CN104104698A CN 104104698 A CN104104698 A CN 104104698A CN 201310115413 A CN201310115413 A CN 201310115413A CN 104104698 A CN104104698 A CN 104104698A
- Authority
- CN
- China
- Prior art keywords
- data
- data buffer
- web
- fragment
- buffer storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Transfer Between Computers (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a Web data cache processing method, device and system. The Web data cache processing method comprises the steps that data transmitted back by a target server are acquired; whether the data are data needed to be cached is judged according to a protocol type of the data, and if the judgment result is yes, the data are cached to a data cache region, or the data are not cached. An anticipating technology is adopted by the Web data cache processing method, and non-cache application data are filtered in advance according to the protocol types so that an objective that forwarding of the non-cache application data is not influenced is achieved.
Description
Technical field
The present invention relates to web data processing field, particularly a kind of Web data buffer storage processing method, device and system.
Background technology
Web buffer memory of the prior art is that all data are carried out to buffer memory, there is no deterministic process, affects the forwarding of non-buffer memory application data.
Summary of the invention
The invention provides a kind of Web data buffer storage processing method, device and system that non-buffer memory application data forwards that do not affect.
For addressing the above problem, as first aspect of the present invention, provide a kind of Web data buffer storage processing method, comprising: the data of obtaining destination server be passed back; Judge according to the protocol type of data whether these data are the data that need buffer memory, if so, by data buffer storage to data buffer area, otherwise buffer memory not.
Further, method also comprises: obtain the access request from user; File destination name in read access request, and judge whether the corresponding data of file destination name have been buffered in data buffer area; If be buffered, the URL in access request is redirected in data buffer area with the corresponding data of file destination name, and these data are directly sent to user; Otherwise, user and corresponding destination server are linked again.
Further, method also comprises: obtain from the fragment data of destination server and fragment data is cached to data buffer area, fragment data is recombinated to form complete data.
Further, method also comprises: obtain from the fragment data of destination server and fragment data is cached to data buffer area; Obtain that user drags video progress and the fragment data access request that forms judges whether the corresponding data of fragment data access request have been buffered in data buffer area; If be buffered, the URL in fragment data access request is redirected in data buffer area with the corresponding data of fragment data access request, and these data are directly sent to user.
Further, method also comprises: the corresponding data of time period before downloading slicing data access request automatically, and be buffered in data buffer area.
Further, be reassembled into partial data by automatically downloading the data that obtain and the fragment data being buffered in data buffer area.
As second aspect of the present invention, a kind of Web data buffer storage processing unit is provided, comprising: receiver module, obtains the data of destination server be passed back; Cache module, judges according to the protocol type of data whether these data are the data that need buffer memory, if so, by data buffer storage to data buffer area, otherwise buffer memory not.
Further, also comprise: fragment data recombination module, the fragment data being cached in data buffer area is recombinated to form complete data.
As the 3rd aspect of the present invention, a kind of Web data buffer storage treatment system is provided, it is characterized in that, comprising: client; Web data buffer storage processing unit, it is above-mentioned Web data buffer storage processing unit, Web data buffer storage processing unit comprises WAN mouth and LAN mouth; WAN mouth is connected with client; Host exchange, is connected with LAN mouth.
Because the present invention adopts anticipation technology, filter out in advance non-buffer memory application data according to protocol type, therefore, reach and do not affected the object that non-buffer memory application data forwards.
Brief description of the drawings
Fig. 1 schematically shows the schematic diagram of the Web data buffer storage processing method in the present invention;
Fig. 2 schematically shows a caching process process schematic diagram in embodiment;
Fig. 3 schematically shows handshake procedure schematic diagram;
Fig. 4 schematically shows the caching process process schematic diagram in another embodiment; And
Fig. 5 schematically shows the frame diagram of Web data buffer storage treatment system.
Embodiment
Below in conjunction with accompanying drawing, embodiments of the invention are elaborated, but the multitude of different ways that the present invention can be defined by the claims and cover is implemented.
As a first aspect of the present invention, a kind of Web data buffer storage processing method is provided, comprising: the data of obtaining destination server be passed back; Judge according to the protocol type of data whether these data are the data that need buffer memory, if so, by data buffer storage to data buffer area, otherwise buffer memory not.
Please refer to Fig. 1, when the destination server of accessing outer network from inner network, when the destination server of outer net sends data return, meeting of the present invention first such as, judges according to data protocol type (http agreement etc.) whether such data are the data that need buffer memory.If need buffer memory, adopt and for example dynamically read mirror image technology, needing the deposit data of buffer memory in data buffer area (: the hard disk in the hard disk cache server that buffer memory is used).Because the present invention adopts anticipation technology, filter out in advance non-buffer memory application data according to protocol type, therefore, reach and do not affected the object that non-buffer memory application data forwards.
In addition, video, webpage, picture that the present invention can be used for based on to web carry out buffer memory, so that user while again accessing identical content, does not take outer net bandwidth, directly such as, from data buffer area (caching server etc.), extract associated documents.
Preferably, the method also comprises: obtain the access request from user; File destination name in read access request, and judge whether the corresponding data of file destination name have been buffered in data buffer area; If be buffered, the URL in access request be redirected in data buffer area and the corresponding data of file destination name, and these data are directly sent to user; Otherwise, user and corresponding destination server are linked again.
Please refer to Fig. 2, in the time of customer access network, data can be processed as shown in Figure 2.The get packet that user sends is understood the squid using in first and Web data buffer storage processing method and is established the link, so then the file destination name of get in Web cache read data fetch packet inquires about in data cached district.
If inquiring the data of user's access is (being to deposit this data in data cached district) being buffered, squid can be redirected to the document location at file destination place in data buffer area the URL of user's get bag so, and file data is directly sent to terminal use.
Please refer to Fig. 3, user is with http mode accesses network time, all can be first and server establish the link by the mode of three-way handshake.Then being only data transmits mutually.
If squid is to the file destination name inquiry in get message when data cached, squid can re-start the virtual connection of shaking hands for 3 times according to the target that in this session link, 3 times are shaken hands, and then allows terminal use and destination server again link.At this moment user's data can directly read from outer network server.The data of coming if read back meet buffer memory requirement, and squid can carry out buffer memory to these data so.Because this part is transparent to user, so these data manipulations are to affect any internet behavior of user.
The development with rapid changepl. never-ending changes and improvements of data format in the Internet, mainstream data form of the prior art can allow traditional caching server feel simply helpless, the web video that web buffer memory of the prior art cannot the current main flow of buffer memory.
The present invention has also solved in prior art the technical problem of file that cannot the current main flow of buffer memory non-buffer memory form by following mode.
Preferably, the method also comprises: obtain from the fragment data of destination server and fragment data is cached to data buffer area, fragment data is recombinated to form complete data.Are burst download formats owing to there being a lot of video files in prior art, for so non-buffer memory formatted file, the present invention has adopted above-mentioned IP fragmentation and reassembly technology.
Please refer to Fig. 4, when finding that packet is the file of burst download formats, web buffer memory can get off these packet segment buffer memorys, then adopts IP fragmentation and reassembly technology, according to the sequence number of file fragment, these file fragments are carried out to IP fragmentation and reassembly, makes it to become complete video file.
For the non-cache file that drags video progress due to terminal use and cause, meeting of the present invention first changes into fragment data these data, then carries out buffer memory, finally adopts IP fragmentation and reassembly technology to recombinate to these data again.For this reason, preferably, the method also comprises: obtain from the fragment data of destination server and fragment data is cached to data buffer area; Obtain that user drags video progress and the fragment data access request that forms judges whether the corresponding data of fragment data access request have been buffered in data buffer area; If be buffered, the URL in fragment data access request is redirected in data buffer area with the corresponding data of fragment data access request, and these data are directly sent to user.
Preferably, the method also comprises: the corresponding data of time period before downloading slicing data access request automatically, and be buffered in data buffer area.Preferably, be reassembled into partial data by automatically downloading the data that obtain and the fragment data being buffered in data buffer area.
As Fig. 4, if in caching server a buffer memory this part data of 31-70, when user is in the time watching this file video of A, start to watch from starting directly to drag to 31-40, web buffer memory can first be redirected to chaining of user this section of 31-70 so, allows user first load this part video file.Then server can automatically link to original server and gets on to download this part data of 0-30, then all fragment datas is integrated and is left in caching server, to load partial data when next time other terminal access.
Especially, the present invention belongs to same video but the video of different filenames by these, adopts URL redirect relevance technology to carry out buffer memory, and cache hit rate can reach more than 90%.
As a second aspect of the present invention, a kind of Web data buffer storage processing unit is provided, it is characterized in that, comprising: receiver module, obtains the data of destination server be passed back; Cache module, judges according to the protocol type of data whether these data are the data that need buffer memory, if so, by data buffer storage to data buffer area, otherwise buffer memory not.
Preferably, also comprise: fragment data recombination module, the fragment data being cached in data buffer area is recombinated to form complete data.
As a third aspect of the present invention, a kind of Web data buffer storage treatment system is provided, it is characterized in that, comprising: client; Web data buffer storage processing unit, it is above-mentioned Web data buffer storage processing unit, Web data buffer storage processing unit comprises WAN mouth and LAN mouth; WAN mouth is connected with client; Host exchange, is connected with LAN mouth.
Please refer to Fig. 5, the Web data buffer storage processing unit in the present invention is deployed in the middle of router and host exchange, disposes by the pattern of bridge joint.Be that Web data buffer storage processing unit is erected between router and host exchange.If WAN mouth and LAN mouth have been connect instead, Web data buffer storage processing unit is to affect the whole network operation, just there is no buffer memory effect.
On the present invention's web caching technology in the prior art, carry out secondary development, added the redirect of URL relevance, burst to download restructuring, turn the cache way such as IP fragmentation and reassembly.Carry out caching process mainly for non-buffer memory form and non-mainstream form web video and web page files, picture, make the web video of the buffer memory main flow that the present invention can be correct.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.
Claims (9)
1. a Web data buffer storage processing method, is characterized in that, comprising:
Obtain the data of destination server be passed back;
Judge according to the protocol type of described data whether these data are the data that need buffer memory, if so, by described data buffer storage to data buffer area, otherwise buffer memory not.
2. Web data buffer storage processing method according to claim 1, is characterized in that, described method also comprises:
Obtain the access request from user;
Read the file destination name in described access request, and judge whether the corresponding data of described file destination name have been buffered in described data buffer area;
If be buffered, the URL in described access request is redirected in described data buffer area with the corresponding data of described file destination name, and these data are directly sent to described user;
Otherwise, described user and corresponding destination server are linked again.
3. Web data buffer storage processing method according to claim 1, is characterized in that, described method also comprises:
Obtain from the fragment data of destination server and by described fragment data and be cached to described data buffer area, described fragment data is recombinated to form complete data.
4. Web data buffer storage processing method according to claim 1, is characterized in that, described method also comprises:
Obtain from the fragment data of destination server and by described fragment data and be cached to described data buffer area;
Obtain that user drags video progress and the fragment data access request that forms judges whether the corresponding data of described fragment data access request have been buffered in described data buffer area;
If be buffered, the URL in described fragment data access request is redirected in described data buffer area with the corresponding data of described fragment data access request, and these data are directly sent to described user.
5. Web data buffer storage processing method according to claim 4, is characterized in that, described method also comprises:
Automatically download described fragment data access request corresponding data of time period before, and be buffered in described data buffer area.
6. Web data buffer storage processing method according to claim 5, is characterized in that, the data that described automatic download is obtained are reassembled into partial data with the fragment data being buffered in described data buffer area.
7. a Web data buffer storage processing unit, is characterized in that, comprising:
Receiver module, obtains the data of destination server be passed back;
Cache module, judges according to the protocol type of described data whether these data are the data that need buffer memory, if so, by described data buffer storage to data buffer area, otherwise buffer memory not.
8. Web data buffer storage processing unit according to claim 7, is characterized in that, also comprises:
Fragment data recombination module, recombinates to form complete data by the fragment data being cached in described data buffer area.
9. a Web data buffer storage treatment system, is characterized in that, comprising:
Client;
Web data buffer storage processing unit, it is the Web data buffer storage processing unit described in claim 7 or 8, described Web data buffer storage processing unit comprises WAN mouth and LAN mouth; Described WAN mouth is connected with described client;
Host exchange, is connected with described LAN mouth.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310115413.XA CN104104698A (en) | 2013-04-01 | 2013-04-01 | Web data cache processing method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310115413.XA CN104104698A (en) | 2013-04-01 | 2013-04-01 | Web data cache processing method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104104698A true CN104104698A (en) | 2014-10-15 |
Family
ID=51672498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310115413.XA Pending CN104104698A (en) | 2013-04-01 | 2013-04-01 | Web data cache processing method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104104698A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021569A (en) * | 2016-05-31 | 2016-10-12 | 广东能龙教育股份有限公司 | Method and system for solving Hibernate distributed data caching |
| CN106202082A (en) * | 2015-04-30 | 2016-12-07 | 阿里巴巴集团控股有限公司 | The method and device of built-up foundation data buffer storage |
| CN108702511A (en) * | 2017-07-28 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Method, equipment and the system of transmission of video |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101119385A (en) * | 2007-08-10 | 2008-02-06 | 深圳市深信服电子科技有限公司 | Method for enhancing HTTP network velocity using WebPush |
| US7412535B2 (en) * | 2001-12-19 | 2008-08-12 | International Business Machines Corporation | Method and system for caching fragments while avoiding parsing of pages that do not contain fragments |
| CN101997927A (en) * | 2010-11-18 | 2011-03-30 | 厦门市美亚柏科信息股份有限公司 | Method and system for caching data of WEB platform |
| CN102244644A (en) * | 2010-05-11 | 2011-11-16 | 华为技术有限公司 | Method and device for releasing multimedia file |
| CN102594877A (en) * | 2012-01-19 | 2012-07-18 | 网宿科技股份有限公司 | Method and system for combining redirected download request and agency service to accelerate network service |
-
2013
- 2013-04-01 CN CN201310115413.XA patent/CN104104698A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7412535B2 (en) * | 2001-12-19 | 2008-08-12 | International Business Machines Corporation | Method and system for caching fragments while avoiding parsing of pages that do not contain fragments |
| CN101119385A (en) * | 2007-08-10 | 2008-02-06 | 深圳市深信服电子科技有限公司 | Method for enhancing HTTP network velocity using WebPush |
| CN102244644A (en) * | 2010-05-11 | 2011-11-16 | 华为技术有限公司 | Method and device for releasing multimedia file |
| CN101997927A (en) * | 2010-11-18 | 2011-03-30 | 厦门市美亚柏科信息股份有限公司 | Method and system for caching data of WEB platform |
| CN102594877A (en) * | 2012-01-19 | 2012-07-18 | 网宿科技股份有限公司 | Method and system for combining redirected download request and agency service to accelerate network service |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202082A (en) * | 2015-04-30 | 2016-12-07 | 阿里巴巴集团控股有限公司 | The method and device of built-up foundation data buffer storage |
| CN106202082B (en) * | 2015-04-30 | 2020-01-14 | 菜鸟智能物流控股有限公司 | Method and device for assembling basic data cache |
| CN106021569A (en) * | 2016-05-31 | 2016-10-12 | 广东能龙教育股份有限公司 | Method and system for solving Hibernate distributed data caching |
| CN108702511A (en) * | 2017-07-28 | 2018-10-23 | 深圳市大疆创新科技有限公司 | Method, equipment and the system of transmission of video |
| WO2019019128A1 (en) * | 2017-07-28 | 2019-01-31 | 深圳市大疆创新科技有限公司 | Video transmission method, device, and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1886472B1 (en) | Method for multipart encoding | |
| US20060168129A1 (en) | System and method for enhancing network browsing speed by setting a proxy server on a handheld device | |
| EP1829284B1 (en) | System and method for enhancing network-browsing speed by setting a proxy server on a handheld device | |
| US9237210B2 (en) | Internet access method, terminal and storage medium | |
| CN104580192B (en) | The treating method and apparatus of the network access request of application program | |
| CN103338249B (en) | Caching method and device | |
| US20020062384A1 (en) | Method of proxy-assisted predictive pre-fetching | |
| WO2016127799A1 (en) | Video advertisement filter method, apparatus and system | |
| US20120054295A1 (en) | Method and apparatus for providing or acquiring the contents of a network resource for a mobile device | |
| US20060259690A1 (en) | Methods and system for prepositioning frequently accessed web content | |
| US20090094377A1 (en) | Method and system for accelerating browsing sessions | |
| CN105760405A (en) | Methods and devices for obtaining network resource for loading web page and method and device for caching network resource | |
| JP2009157444A (en) | Service providing system, gateway, and server | |
| US20150026289A1 (en) | Content source discovery | |
| US20060009215A1 (en) | Storing data content in a remote computer | |
| WO2012113176A1 (en) | Method and system for displaying web pages on terminal | |
| WO2016062077A1 (en) | Method and apparatus for redirection to web page | |
| CN104104698A (en) | Web data cache processing method, device and system | |
| JP6205765B2 (en) | VIDEO DISTRIBUTION DEVICE, VIDEO DISTRIBUTION PROGRAM, VIDEO DISTRIBUTION METHOD, AND VIDEO DISTRIBUTION SYSTEM | |
| JP2003288261A (en) | Data transferring device, data transferring method, and program | |
| WO2013113150A1 (en) | Cache optimization method, cache and cache optimization system | |
| CN103686852B (en) | A kind of method, equipment and Wireless acceleration system that interaction data is processed | |
| US20140089468A1 (en) | Request-agnostic caching for a data resource collection | |
| Schmitt et al. | Internet media upload caching for poorly-connected regions | |
| EP2552082B1 (en) | Favourite web site acceleration method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518000 Guangdong city of Shenzhen province Baoan District Longhua Street tabulaeformis business building 1801-1805 (Office) Applicant after: Shenzhen alliance Polytron Technologies Inc Address before: 518000 Guangdong city of Shenzhen province Baoan District Longhua Street tabulaeformis business building 1801-1805 (Office) Applicant before: SHENZHEN WAYOS SCIENCE & TECHNOLOGY CO., LTD. |
|
| COR | Change of bibliographic data | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20141015 |