CN104750743A - System and method for ticking and rechecking transaction files - Google Patents
System and method for ticking and rechecking transaction files Download PDFInfo
- Publication number
- CN104750743A CN104750743A CN201310749178.1A CN201310749178A CN104750743A CN 104750743 A CN104750743 A CN 104750743A CN 201310749178 A CN201310749178 A CN 201310749178A CN 104750743 A CN104750743 A CN 104750743A
- Authority
- CN
- China
- Prior art keywords
- file
- transaction record
- source
- son
- storage space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention provides a system and a method for ticking and rechecking transaction files. The system comprises a file splitting device and a file ticking and rechecking device. The file splitting device is used for respectively splitting source files and object files to obtain source file split sets and object file split sets, the source file split sets comprise N source son files, the object file split sets comprise N object son files, the N is a natural number larger than 1, the file ticking and rechecking device is used for ticking and rechecking first source son files and first object son files which correspond to one another, and transaction records in the first source son files and the first object son files are mapped into preliminarily configured storage spaces so as to be compared to one another.
Description
Technical field
The present invention relates to transaction file to hook system and method.
Background technology
At present, the industries such as finance, telecommunications, internet all can check by pen the transaction details (such as, comprising dealing money, service charge, user name, account number, transaction time of origin etc.) occurred, and it is right that this class process is referred to as transaction hook.Transaction hooks for verifying to transaction acceptance and uniformity for the treatment of between different system, thus promotes the reliability of data processed result.
Along with the development of every profession and trade, portfolio is increased sharply, and the number of transaction that every day occurs has reached necessarily even hundred million ranks, and system, when carrying out automated transaction and blending, is produced consuming time more and more longer.Hooking right scheme to the detail of two transaction files (such as, file A and file B) at present comprises following several:
One, sort respectively to the transaction record (Flow Record) in A and B two files, then order reads and carries out record comparison.
The shortcoming of this scheme is as follows:
1) file ordering is more consuming time;
2) the great file of data volume (such as 100G) cannot be loaded into internal memory, also needs to consider external sort;
3) higher memory headroom is needed in sequencer procedure;
4) one process is consuming time larger;
Two, travel through a file, and each transaction record is searched in another file.
The shortcoming of this scheme is as follows:
1) although do not need to sort, each transaction all will be searched in big data quantity, seriously consuming time;
2) the great file of data volume directly cannot be loaded into internal memory, searches more complicated;
3) if the full dose inconsistent data to two files will be hooked, not only to do A file transactions at B ff, also need to do B file record again at A ff, still consuming time;
4) one process is consuming time larger
Three, file A and file B is loaded onto in database table, then carries out hook with data base table recording association right.
The shortcoming of this scheme is as follows:
1) need to do load of file, consuming time more when data volume is large;
2) usage data storehouse is needed, not easily multiplexing and migration;
3) storage space is expended;
summary of the invention
For overcoming one or more shortcoming, the invention provides transaction file and hooking system and method.
According to an aspect of the present invention, described transaction file comprises source file and file destination, this system comprises file division device and file hooks device, wherein, described file division device is used for splitting respectively source file and file destination, obtain the source file segmentation set be made up of N number of source son file and the file destination segmentation set be made up of N number of target son file, N be greater than 1 natural number, described file hooks device for hooking the first corresponding source son file and first object son file, wherein the transaction record in this first source son file and first object son file is mapped to pre-configured storage space to compare.
According to an aspect of the present invention, described transaction file comprises source file and file destination, the method comprises: split respectively source file and file destination, obtain the source file segmentation set be made up of N number of source son file and the file destination segmentation set be made up of N number of target son file, N be greater than 1 natural number, hook the first corresponding source son file and first object son file, wherein the transaction record in this first source son file and first object son file is mapped to pre-configured storage space to compare.
Advantages more of the present invention comprise following at least one:
The different big data quantities detailed chronological file of this motion to different system or CMOS macro cell uses Hash means to be split as multiple son file, then the son file transaction hook of the same Hash result of multi-course concurrency process is right.Hook not usage data storehouse in process, and without the need to sequence, only travel through file transactions record once.
Use native system, contrast with legacy system, have following advantage: the structure of lightweight, be easy to multiplexing; Without the need to usage data storehouse, be easy to migration; Internal memory uses less; Hook to process without the need to sequence, system handling property obtains height.
Accompanying drawing explanation
After having read the specific embodiment of the present invention with reference to accompanying drawing, those skilled in the art will become apparent various aspects of the present invention.One skilled in the art will appreciate that these accompanying drawings only for coordinating embodiment that technical scheme of the present invention is described, and and be not intended to be construed as limiting protection scope of the present invention.
Fig. 1 is the schematic diagram of transaction file hook to system according to an aspect of the present invention.
Fig. 2 hooks operation chart according to the transaction file of an example of the present invention.
Fig. 3 is the schematic diagram of transaction file hook to method according to an aspect of the present invention.
Embodiment
With reference to the accompanying drawings, the specific embodiment of the present invention is described in further detail.In the following description, in order to the object explained, state many details to provide the thorough understanding of the one or more aspects to embodiment.But, can it is evident that for those skilled in the art, the less degree of these details can come one or more aspects of practicing various embodiments.Therefore the description below is not regarded as circumscribed, but limits protection domain by claims.
Fig. 1 is the schematic diagram of transaction file hook to system according to an aspect of the present invention.As shown in Figure 1, transaction file hooks and can comprise file division device and file to system and hook device.File division device is used for splitting respectively source file and file destination, and the source file segmentation set obtaining be made up of N number of source son file and file destination be made up of N number of target son file are split and gathered, N be greater than 1 natural number.Transaction record in this first source son file and first object son file, is wherein mapped to pre-configured storage space to compare for hooking the first corresponding source son file and first object son file to device by file hook.
Fig. 2 hooks operation chart according to the transaction file of an example of the present invention.In this example, file division device is split source file and file destination respectively, wherein source file is divided into N number of source son file, file destination is divided into N number of target son file, forms source son file set and the set of target son file.On the other hand, for source son file set and the set of target son file, file hooks and such as hooks comparing by source son file 1 and target son file 1 to device, carrying out hooking source son file 2 and target son file 2 to comparing, being carried out hooking comparing by source son file N and target son file N.As mentioned above, carried out the transaction record hooked in two right son files and can be mapped to pre-configured storage space to compare, finally produced feedback file according to comparison result.
In one example, file division device is split source file and file destination respectively according to transaction record serial number, and wherein source file is mapped to corresponding source son file and target son file respectively with the identical transaction record serial number in file destination.
Such as, can to comprise transaction record as follows for source file:
Serial number exchange hour dealing money
010123 20130515 780
029140 20130515 1290
005910 20130515 90
781311 20130515 400190
035910 20130515 5000
000911 20130515 120
117000 20130515 2980
200100 20130515 12000
600143 20130515 200
002409 20130515 10
011901 20130515 3800
430007 20130515 9100
After file division, following source file segmentation set can be generated:
File F_01
029140 20130515 1290
000911 20130515 120
200100 20130515 12000
002409 20130515 10
File F_02
010123 20130515 780
035910 20130515 5000
600143 20130515 200
011901 20130515 3800
File F_03
005910 20130515 90
781311 20130515 400190
117000 20130515 2980
430007 20130515 9100
Identical process is also used for file destination.Be understandable that, source file is mapped to corresponding source son file and target son file respectively with the identical transaction record serial number in file destination, but upon splitting, source son file may comprise not identical with the transaction record serial number in target son file transaction record.
In one example, file division device uses HASH algorithm that the transaction record in source file and file destination is mapped to corresponding source son file and target son file respectively.Here, HASH algorithm can using transaction record serial number as map source, using son file as mapping result.
In one example, the storage space element of pre-configured storage space is transaction record, and file hooks and is configured to device: compared with the identical position that the first transaction record of first object son file is mapped to storage space by first transaction record of the first source son file with same transaction record serial number.Here, file hooks and also can use HASH algorithm that first transaction record of the first source son file with same transaction record serial number and the first transaction record of first object son file are mapped to the identical position of storage space to device.
In another example, file hooks and is configured to device:
Read the transaction record in the first source son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, in this position store transaction logs,
Read the transaction record in first object son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, is added to this corresponding position by after the value negate of transaction record.
As mentioned above, first transaction record of first transaction record and first object son file with the first source son file of same transaction record serial number will have identical position, therefore after the value of the transaction record in the value of the transaction record in the source son file of this same position and target son file being added, the result obtained is stored in this same position, as the hook of the transaction record serial number indicated by this same position to result.
Transaction file according to certain aspects of the invention hooks can also comprise feedback assembly to system, feedback assembly judge file hook to device complete hook to after the value of storage space element of storage space whether be 0, be not wherein the transaction record write feedback file of 0 by the value of storage space element, as hook to unsuccessful transaction record.Such as, the value of the transaction record dealing money in the son file of source is 100, the value of the dealing money of the transaction record in target son file is 120, then according to above-mentioned example, the initial value that particular memory location stores is 100, to be-120 after value 120 negate of the dealing money of the transaction record in target son file, finally the value of this particular memory location be-20, and it represents that transaction hooks failure.
Hook efficiency for increasing, file hooks that M process can be used to process the hook of corresponding source son file in N number of source son file and N number of target son file and target son file respectively to device right, M be greater than 1 natural number.In one example, M can equal N.
Above-mentioned storage space can be such as internal memory, and preferably storage space is the array in internal memory.In this case, situation about hooking the first source son file and first object son file is described.In this example, first open up one section of memory headroom, represent with array M, length is L, and array element is transaction record; Open source son file and target son file; Read a transaction record of source son file, use HASH algorithm according to this according to transaction record serial number, mould is L, obtains the position of this transaction record in array, i.e. array index D; By transaction record assignment to array element D; According to above-mentioned steps, after giving corresponding array element by the All Activity record assignment of source son file, close source son file.Similarly, a transaction record of target son file is read; Use HASH algorithm according to this according to transaction record serial number, mould is L, obtains the position of this transaction record in array, i.e. array index D; Array element D is added to by after transaction record (such as, dealing money) wherein negate; According to above-mentioned steps, the All Activity record of target son file is added up to after corresponding array element, close target son file; Then feedback file is opened with additional WriteMode; Traversal array M, order obtains the array element in array M, if wait that hooking data is 0 wherein in array element, then represents that this stream transaction record hooks success; Otherwise unsuccessful, by transaction record serial number write feedback file; Close feedback file, so far hook end.
In above-mentioned situation, the L of array is greater than the transaction record quantity in source son file or target son file.But as mentioned above, source son file may comprise not identical with the transaction record serial number in target son file transaction record.Therefore, L can be transaction record quantity in the first source son file and first object son file and.
Fig. 3 is the schematic diagram of transaction file hook to method according to an aspect of the present invention.As shown in the figure, the method comprises:
Split respectively source file and file destination, the source file segmentation set obtaining being made up of N number of source son file and file destination segmentation be made up of N number of target son file are gathered, N be greater than 1 natural number,
Hook the first corresponding source son file and first object son file, wherein the transaction record in this first source son file and first object son file is mapped to pre-configured storage space to compare.
In one example in which, split respectively source file and file destination according to transaction record serial number, wherein source file is mapped to corresponding source son file and target son file respectively with the identical transaction record serial number in file destination.
In one example in which, use HASH algorithm that the transaction record in source file and file destination is mapped to corresponding source son file and target son file respectively.
In one example in which, the storage space element of pre-configured storage space is transaction record, and the method comprises: compared with the identical position that the first transaction record of first object son file is mapped to storage space by first transaction record of the first source son file with same transaction record serial number.
In one example in which, use HASH algorithm that first transaction record of the first source son file with same transaction record serial number and the first transaction record of first object son file are mapped to the identical position of storage space.
In one example in which, read the transaction record in the first source son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, in this position store transaction logs, read the transaction record in first object son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, is added to this corresponding position by after the value negate of transaction record.Such as, the position of the transaction record HASH of in the son file of source to internal memory, data count A, a transaction record HASH in target son file is to the identical position of internal memory, data are designated as-A, after accumulation operations, if the address values of this internal memory is A+(-A)=0, then think and hook coupling to success.
In one example in which, the method also comprises:
Judge file hook to device complete hook to after the value of storage space element of storage space whether be 0, be not wherein the transaction record write feedback file of 0 by the value of storage space element, as hook to unsuccessful transaction record.
In one example in which, use M process to process the hook of corresponding source son file in N number of source son file and N number of target son file and target son file respectively right, M be greater than 1 natural number.
In one example in which, storage space is internal memory.Such as, storage space is the array in internal memory.Again such as, the length of array is L, wherein L be transaction record quantity in the first source son file and first object son file and.
By the description of above embodiment, those skilled in the art can understand, and when without departing from the spirit and scope of the present invention, can also do various change and replacement to the specific embodiment of the present invention.These change and replace and all drop in claims of the present invention limited range.
Claims (22)
1. transaction file hooks a system, it is characterized in that,
Described transaction file comprises source file and file destination,
This system comprises file division device and file hooks device, wherein,
Described file division device is used for splitting respectively source file and file destination, and the source file segmentation set obtaining be made up of N number of source son file and file destination be made up of N number of target son file are split and gathered, N be greater than 1 natural number,
Transaction record in this first source son file and first object son file, is wherein mapped to pre-configured storage space to compare for hooking the first corresponding source son file and first object son file to device by described file hook.
2. the system as claimed in claim 1, is characterized in that,
Described file division device is split source file and file destination respectively according to transaction record serial number, and wherein source file is mapped to corresponding source son file and target son file respectively with the identical transaction record serial number in file destination.
3. system as claimed in claim 2, is characterized in that,
Described file division device uses HASH algorithm that the transaction record in source file and file destination is mapped to corresponding source son file and target son file respectively.
4. system as claimed in claim 3, is characterized in that,
The storage space element of described pre-configured storage space is transaction record,
Described file hooks and is configured to device:
First transaction record of the first source son file with same transaction record serial number is compared with the identical position that the first transaction record of first object son file is mapped to storage space.
5. system as claimed in claim 4, is characterized in that,
Described file hooks and uses HASH algorithm that first transaction record of the first source son file with same transaction record serial number and the first transaction record of first object son file are mapped to the identical position of storage space to device.
6. the system as described in claim 4 or 5, is characterized in that,
Described file hooks and is configured to device:
Read the transaction record in the first source son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, in this position store transaction logs,
Read the transaction record in first object son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, is added to this corresponding position by after the value negate of transaction record.
7. system as claimed in claim 6, it is characterized in that, also comprise feedback assembly, described feedback assembly judge described file hook to device complete hook to after the value of storage space element of storage space whether be 0, be not wherein the transaction record write feedback file of 0 by the value of storage space element, as hook to unsuccessful transaction record.
8. the system as claimed in claim 1, is characterized in that,
Described file hooks that to use M process to process the hook of corresponding source son file in N number of source son file and N number of target son file and target son file respectively to device right, M be greater than 1 natural number.
9. the system as claimed in claim 1, is characterized in that,
Described storage space is internal memory.
10. system as claimed in claim 9, is characterized in that,
Described storage space is the array in internal memory.
11. systems as claimed in claim 10, is characterized in that,
The length of described array is L, wherein L be transaction record quantity in the first source son file and first object son file and.
12. 1 kinds of transaction files hook method, it is characterized in that,
Described transaction file comprises source file and file destination,
The method comprises:
Split respectively source file and file destination, the source file segmentation set obtaining being made up of N number of source son file and file destination segmentation be made up of N number of target son file are gathered, N be greater than 1 natural number,
Hook the first corresponding source son file and first object son file, wherein the transaction record in this first source son file and first object son file is mapped to pre-configured storage space to compare.
13. methods as claimed in claim 12, is characterized in that,
Split respectively source file and file destination according to transaction record serial number, wherein source file is mapped to corresponding source son file and target son file respectively with the identical transaction record serial number in file destination.
14. methods as claimed in claim 12, is characterized in that,
Use HASH algorithm that the transaction record in source file and file destination is mapped to corresponding source son file and target son file respectively.
15. methods as claimed in claim 14, is characterized in that,
The storage space element of described pre-configured storage space is transaction record,
The method comprises:
First transaction record of the first source son file with same transaction record serial number is compared with the identical position that the first transaction record of first object son file is mapped to storage space.
16. methods as claimed in claim 14, is characterized in that,
Use HASH algorithm that first transaction record of the first source son file with same transaction record serial number and the first transaction record of first object son file are mapped to the identical position of storage space.
17. methods as described in claim 15 or 16, is characterized in that,
Read the transaction record in the first source son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, in this position store transaction logs,
Read the transaction record in first object son file, according to transaction record serial number, transaction record is mapped to the corresponding position of storage space, is added to this corresponding position by after the value negate of transaction record.
18. methods as claimed in claim 17, is characterized in that, also comprise:
Judge described file hook to device complete hook to after the value of storage space element of storage space whether be 0, be not wherein the transaction record write feedback file of 0 by the value of storage space element, as hook to unsuccessful transaction record.
19. methods as claimed in claim 12, is characterized in that,
Use M process to process the hook of corresponding source son file in N number of source son file and N number of target son file and target son file respectively right, M be greater than 1 natural number.
20. methods as claimed in claim 12, is characterized in that,
Described storage space is internal memory.
21. methods as claimed in claim 20, is characterized in that,
Described storage space is the array in internal memory.
22. methods as claimed in claim 21, is characterized in that,
The length of described array is L, wherein L be transaction record quantity in the first source son file and first object son file and.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310749178.1A CN104750743A (en) | 2013-12-31 | 2013-12-31 | System and method for ticking and rechecking transaction files |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310749178.1A CN104750743A (en) | 2013-12-31 | 2013-12-31 | System and method for ticking and rechecking transaction files |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104750743A true CN104750743A (en) | 2015-07-01 |
Family
ID=53590440
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310749178.1A Pending CN104750743A (en) | 2013-12-31 | 2013-12-31 | System and method for ticking and rechecking transaction files |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104750743A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930630A (en) * | 2015-09-15 | 2016-09-07 | 中国银联股份有限公司 | Data analysis method and apparatus |
| CN106960052A (en) * | 2017-03-31 | 2017-07-18 | 深圳微众税银信息服务有限公司 | A kind of collage-credit data acquisition method and system |
| CN107766458A (en) * | 2017-09-27 | 2018-03-06 | 中国银行股份有限公司 | File crashes method, apparatus, computer equipment and computer-readable recording medium |
| CN107993147A (en) * | 2017-11-13 | 2018-05-04 | 中国银行股份有限公司 | The remaining sum control method and device of hot spot account |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7428540B1 (en) * | 2000-03-03 | 2008-09-23 | Intel Corporation | Network storage system |
| CN101957858A (en) * | 2010-09-27 | 2011-01-26 | 中兴通讯股份有限公司 | Data comparison method and device |
| CN101989929A (en) * | 2010-11-17 | 2011-03-23 | 中兴通讯股份有限公司 | Disaster recovery data backup method and system |
| CN103379160A (en) * | 2012-04-25 | 2013-10-30 | 上海咏云信息技术有限公司 | Difference synchronizing method for oversized file |
-
2013
- 2013-12-31 CN CN201310749178.1A patent/CN104750743A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7428540B1 (en) * | 2000-03-03 | 2008-09-23 | Intel Corporation | Network storage system |
| CN101957858A (en) * | 2010-09-27 | 2011-01-26 | 中兴通讯股份有限公司 | Data comparison method and device |
| CN101989929A (en) * | 2010-11-17 | 2011-03-23 | 中兴通讯股份有限公司 | Disaster recovery data backup method and system |
| CN103379160A (en) * | 2012-04-25 | 2013-10-30 | 上海咏云信息技术有限公司 | Difference synchronizing method for oversized file |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930630A (en) * | 2015-09-15 | 2016-09-07 | 中国银联股份有限公司 | Data analysis method and apparatus |
| CN106960052A (en) * | 2017-03-31 | 2017-07-18 | 深圳微众税银信息服务有限公司 | A kind of collage-credit data acquisition method and system |
| CN106960052B (en) * | 2017-03-31 | 2020-09-15 | 深圳微众信用科技股份有限公司 | Credit investigation data acquisition method and system |
| CN107766458A (en) * | 2017-09-27 | 2018-03-06 | 中国银行股份有限公司 | File crashes method, apparatus, computer equipment and computer-readable recording medium |
| CN107993147A (en) * | 2017-11-13 | 2018-05-04 | 中国银行股份有限公司 | The remaining sum control method and device of hot spot account |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103902698B (en) | A kind of data-storage system and storage method | |
| US10496523B2 (en) | Sequential access storage and data de-duplication | |
| CN100468402C (en) | Sort data storage and split catalog inquiry method based on catalog tree | |
| US8849876B2 (en) | Methods and apparatuses to optimize updates in a file system based on birth time | |
| CN102129458B (en) | Method and device for storing relational database | |
| CN103678160B (en) | A kind of method and apparatus of data storage | |
| JP4669067B2 (en) | Dynamic fragment mapping | |
| CN103902702A (en) | Data storage system and data storage method | |
| CN103744628B (en) | SSTable file storage method and device | |
| CN105074724A (en) | Efficient query processing using histograms in a columnar database | |
| CN103164490B (en) | A kind of efficient storage implementation method of not fixed-length data and device | |
| CN102693302A (en) | Quick file comparison method, system and client side | |
| CN103902701A (en) | Data storage system and data storage method | |
| CN103246549B (en) | A kind of method and system of data conversion storage | |
| CN104573022A (en) | Data query method and device for HBase | |
| CN105164673A (en) | Query integration across databases and file systems | |
| CN104298736A (en) | Method and device for aggregating and connecting data as well as database system | |
| CN106227666A (en) | A kind of automated testing method based on big data and system | |
| CN105630475A (en) | Data label organization system and organization method | |
| CN104750743A (en) | System and method for ticking and rechecking transaction files | |
| CN105447035A (en) | Data scanning method and apparatus | |
| CN104615490A (en) | Method and device for data conversion | |
| WO2021016050A1 (en) | Multi-record index structure for key-value stores | |
| CN102521304A (en) | Hash based clustered table storage method | |
| CN106227769A (en) | Date storage method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150701 |
|
| RJ01 | Rejection of invention patent application after publication |