WO2005043529A1

WO2005043529A1 - Watermarking of audio and audio-visual signals

Info

Publication number: WO2005043529A1
Application number: PCT/GB2004/004477
Authority: WO
Inventors: Avi Landenberg
Original assignee: Sonic Arts Limited
Priority date: 2003-10-24
Filing date: 2004-10-22
Publication date: 2005-05-12
Also published as: GB0324892D0; GB2407465A

Abstract

Security (and trackability if security is breached) of limited issue audio and audio visual recordings, e.g. review copies, is enhanced by putting a different audio watermark on to each of the limited issue items, such as CDs or DVDs. The watermark is applied in a two stage process. Apparatus for effecting this automatically is disclosed.

Description

WATERMARKING OF AUDIO AND AUDIO-VISUAL SIGNALS

This invention relates to the watermarking, control and monitoring of multiple copies of audio and audio visual recordings .

Over recent years, the industry devoted to the recording and distribution of audio and audio visual works has undergone a revolution. On the one hand, new technology has made possible the high quality reproduction of audio and audio visual works to a degree of fidelity not achieved previously, in particular by the use of digital sampling and recording techniques. On the other hand, with the rise of such technology came the easy availability of means for copying such audio and audio visual recordings, and of doing so in a way which, in the case of digital recordings, suffered no degradation whatever compared with the original .

In recent years, the recording industry, both purely audio and audio visual, has sought to combat increasing sophistication among the copyists by a variety of means including various copy protection schemes and attacks on the distribution of software which can be used for copying. Separately, the business model of phased release of material has been progressively abandoned, particularly since, as soon as any recording is publicly released, it can be legitimately purchased and then illegitimately copied very rapidly.

Although simultaneous global release is becoming the norm, this does not affect the fact that, prior to release, copies of audio and audio visual works need to be distributed on a confidential or embargoed basis in order that the audio and audio visual publicity industries can be ready at launch time with informed criticism of the work in question. The necessary degree of "pre-release" brings with it particular problems of security, and substantial efforts are taken, particularly with audio or audio visual works likely to command substantial popularity in the marketplace, to ensure that none of the pre-circulated copies can be diverted to an illegitimate copying end. Despite such measures, however, it is clear that interception can take place prior to the delivery of a recording to, e.g. a critic or reviewer, or, once it has been reviewed, the critic or reviewer may not take adequate care to ensure that the recording is secure, and it may be misappropriated, riot necessarily by theft, but, for example, by apparently legitimate loan.

In recent years, so-called watermarking techniques have been developed for enabling' the differentiation between a legitimate recording and a copy or bootleg one . The digital data on a conventional audio compact disc recorded medium may be manipulated in such a way that embedded in the data is a certain pattern of information which can be recognised by appropriate reading equipment, but which, when the compact disc is inserted into a compact disc player, does not adversely affect the reproduction of the recorded work as heard by the listener. Such techniques are widely known and used to identify a particular recording during the pre-mastering stage so that, following the usual CD mastering and manufacturing processes, each of the commercially available CDs bears the same watermark and is accordingly identifiable as genuine using appropriate equipment .

According to the present invention there is provided a method of watermarking audio signals as specified in the claims .

The present invention provides a method for delivering uniquely watermarked audio at higher speeds than hitherto. This is advantageous in itself, but in addition it makes the combination of watermarking and insertion of data to prevent copying fast enough to be a practical proposition.

Watermarks are commonly used in one of two ways . In the first, the watermark identifies the source material, which might be a particular music track. Recipients of the track may read the watermark, and use it to find out more about the track. The second way is for the watermark to identify the recipient. In that case a single music track, for sending to multiple recipients, would be watermarked separately for each recipient.

The present invention relates particularly, though not exclusively, to the second way of watermarking, which involves repeated watermarking of the same material . One example is the batch delivery of audio CDs to known recipients, where each recipient is given a separate watermark ID. A second example is Internet delivery of audio on demand, where the Internet server or local client authenticates the recipient before inserting the recipient watermark into the delivered audio stream.

It is known from practical experience that analogue watermarking is computing intensive. For example, a typical computer transfers digital audio data over a network much faster than it can watermark the same data. Thus the watermarking process can prove a bottleneck, inhibiting f st delivery of watermarked material .

An advantage of the present invention is to reduce the computing overhead of custom watermark delivery. This is achieved as follows. The watermarking is carried out in two passes, the first pass producing part-processed audio, and preferably also producing associated helper data, while each second pass takes the part-processed audio from the first pass and inserts a custom watermark, preferably by simple addition, and preferably utilising the associated helper data from the first pass . It will be appreciated that the purpose of the first pass need not simply be to carry out once a process that otherwise would need to be carried out many times, but rather to process the signal in a way that minimises the computing requirement of the second pass. Even if an audio track is to be watermarked only once, it. helps to do so according to the invention. In practice, the invention can increase the delivery data rate (or alternatively reduce the computing requirement) by a factor of 10.

Depending on circumstances, second passes might happen some days or weeks after the first pass, with the output of the first pass, including helper data, being stored on disk in the interim.

A practical embodiment will now be described, based on the watermarking technique described in EP0245037. This patent describes equipment for labelling audio signals with identification information having an encoder which inserts the binary information into two very narrow notches of center frequencies 2883 and 3417 Hz, between semi-tones in the tonic scale to minimize music breakthrough into the decoding circuits, and to ensure that no fundamental frequencies in the tonic scale will be excluded in the reproduction. The notches are derived from a 3 -stage biquad filter, and are approximately 50 dB deep and 150 Hz wide at the top. The encoder includes a wide bandpass circuit consisting of a 1 KHz highpass filter and a 6 KHz lowpass filter introduced to ensure that the code insertion level is not determined by frequencies, either high or low, which do not adequately mask the code frequencies . The code amplitude is kept a fixed level below the programme, initially adjustable by a suitable control. The code sequence has an addressing pre-amble consisting of a simultaneous burst of both the lower and higher frequencies for a period of 8 digits, followed by a message portion of 40 bits formed of an appropriate stream of the two frequencies .

Although EP0245037 conveys the idea of real-time analogue circuitry, a practical implementation would use digital processing, and the watermarking speed would depend on the computing power available. For the purpose of the present invention, the watermarking process described in EP0245037 can notionally be split into four parts:

1- Notch filtering of the audio. This is done in the first pass, as it is common to all recipients, and is quite computing intensive.

2- Determining when to insert a code. Code insertion points are computed in the first pass, and stored as helper data, for subsequent consumption by the second pass. This can be a great time saver if the codes are widely spaced, as the second pass need not carry out any processing between codes , but merely forwards the audio from the first pass. It is also a good example of the effective use of helper data.

3- Determining the code amplitude profile. The scheme described in the patent requires the code amplitude to be modulated within a single code. The amplitude profile could be computed during the first pass, and stored as helper data, but might require a considerable amount of storage. Alternatively it could be computed during the second pass. The choice is a trade-off between the helper data storage of the first pass, and the computing overhead of the second pass. • Either choice would be compatible with the present invention.

⁴- Inserting the code. Since the code is different for different recipients, part at least of the insertion must be implemented by the second pass . The code preamble mentioned in the patent abstract might be inserted in the first pass, as it is common to all codes.

Helper data should be strongly attached to the first pass audio, so the two do not become separated. In the case of audio that comes as, for example, a . AV file, the file format allows for the appending of proprietary data chunks of arbitrary length. Thus the first pass can store the part-processed audio in conventional format, and append proprietary helper data, all within the same WAV file. The second pass opens the file, reads the helper data, and does the final processing of the audio. Code insertion in the above example should preferably be by simple addition, as this minimises the overhead of the- second pass. Although some watermarking schemes involve complex modulations, it will often be possible to express the final part of the process as an addition. Typically, the audio will consist of 16 bit samples. If the added component turns out to be less than 1 LSB (Least Significant Bit) in amplitude, some low level noise can also be added. It is known to those skilled in the art that such noise, when combined with a signal component much less than 1 LSB, allows the signal component to be accurately added to the digital stream.

The invention is not restricted to watermarking at the time of delivery. In the case of digitally secure music delivery to a PDA (Personal Digital Assistant) , the second pass might happen within the DRM (Digital Rights Management) player of the PDA, each time the music plays . The second pass encoder would be part of a decryption plug-in, with helper data passed to the plug-in within the encrypted digital stream. Depending on the overall system design, the recipient ID might also be passed in the digital stream, or alternatively might be acquired by the plug-in from the secure player, ^' allowing the watermark to identify the player. As PDAs have limited computing power, the advantage of the invention is the reduced computing overhead of the second pass watermark. The present invention thus provides a method of improving security in connection with a limited issue of an audio or audio visual work which comprises making the limited issue by sequentially recording, from a master data carrier, a plurality of individual audio or audio visual recordings, wherein the signal is processed between the master and the recording mechanism to impart to the audio portion of the signal an audio .watermark, the audio watermark being different for each of the plurality of individual audio or audio visual recordings made.

In this way, a limited edition of an audio or audio visual work can be produced where the recording reproduces the audio or audio visual work normally, with no identification or the like being applied to the visual data, but where the audio data contain a watermark enabling each individual recording to be identified. Thus if,, for example, 100 prerelease copies are to be uniquely produced for distribution to the relevant persons, these can be produced in accordance with the invention as 100 recorded units, for example on audio compact disc or Digital Versatile Discs (DVDs) , each of which is unique, though it differs from the others only insofar as the watermark ^'differs between each copy.

In this way, so long as an appropriate record is kept of which specific copy is distributed to whom, if there is any subsequent evidence of improper copying, for example the very rapid appearance of pirate DVDs from offshore, then, by extracting the watermark from the pirate DVD, it can easily be determined from which of the pre-release DVDs the pirated DVD was derived, thus enabling efforts to find out how the counterfeit activity occurred being able to be focussed on the single recipient of the particular DVD rather than having to check through all recipients to try and determine how and where the mis-use occurred.

In a further embodiment, the method of the present invention can be carried out without difficulty using standard equipment controlled by a suitable computer programme . A particular advantage arises in the application of the present invention in terms of the speed with which multiple but individually identifiable copies can be made. The process may be broken down into four steps :

1. analysis of the digital file corresponding to the original audio or audio visual work;

2. creation of a modified file including metadata which^' provides details of where the watermark should be placed,-

3. insertion of the watermark into the intended location, and

4. creation of the final digital file and recording it on to the output medium such as a DVD.

Of these, the first two need only be carried out once, and then steps 3 and 4 are carried out as many times as there are individually identifiable copies needed.

By way of diagrammatic illustration, the method is illustrated in the accompanying drawing which is a block diagram showing how the method can be put into effect. As illustrated, the recorded medium of choice is a DVD, but it will be appreciated that the method of the present invention is applicable to any audio or audio visual recording where the audio data is in a format which can be watermarked in accordance with the known technology.

It should be noted that, if desired, as well as the unique watermarking applied to the audio or audio visual recording, known copy-protection technologies can also be incorporated in each of the limited issue, this further reducing the risk of mis-use of the recordings sent out.

Referring to the drawing, this appears to show a number of discrete elements, but it should be understood that most or all of them can in fact be incorporated in a single computer unit, either stand-alone or networked.

Referring to the drawing, the apparatus for putting the present invention into effect consists essentially of a DVD reader 1 which is connected to a computer mother board 2. Likewise connected to the mother board 2 is a DVD recording unit, or so-called DVD burner 3, and this may be arranged to be fed with a series of blank DVDs for recording by a suitable stack feeder mechanism.

Connected to the computer mother board 2 are a monitor 4, keyboard 5 and disc printer 6, and (not shown) one or more suitable hard discs -containing application programmes which can be run using commands input e.g. via the keyboard 5.

One of the programmes stored on the hard disc(s) of the computer is a programme which can take an audio file and ^a Ply ^a digital watermark to it. This programme is called up by an appropriate overall simple application control programme for carrying out the method of the present invention.

The programme is caused to be automatically activated if a master DVD is placed in the DVD reader 1. This presents the user, on a screen or monitor 4, a menu of options and text boxes which may be filled with appropriate data, either by the user from keyboard 5, or automatically derived from data on the master disc or other file supplied. The user input, however, will include how many physically numbered copies are required, unless this is automatically generated, for example from a "distribution list" file. Once the user has checked that the feeder for the DVD burner 3 has adequate supplies of blank DVDs, the programme is then set to run. First of all, the relevant audio files are extracted from the master DVD and subjected to the application of a digital watermark thereto. The watermarked audio file is then recorded in DVD burner 3 on to a blank DVD together with the visual data from the master disc in DVD reader 1 and, at the end of the recording process, a recorded DVD is ejected from burner 3 into an appropriate outlet hopper. At the same time, the disc printer 6 which is provided with a feed of adhesive label stock, prints data e.g. data corresponding to fixed graphic material as well as the variable data fed to it from the motherboard 2 on to the disc, including a particular serial number, for example 001 for the first disc printed and 050 for the last of a set of 50. As an alternative to the disc printer 6, it is possible to use a standard printer with a supply of label stock suitable for application to the disc once printed. As an alternative it is possible to use a standard printer, or to print directly onto the surface of the disc itself .

Each time the computer causes a fresh blank DVD to be recorded, it incorporates a different digital watermark into the audio data. At the same time, it builds up a correlation between the number of the DVD recording (1 to 50) and the respective digital watermark associated with that number in each case. If the watermark is one which, in a standard reader, produces an alphanumeric string corresponding to that watermark, then a standard printer may, following the manufacture of the last item in the series by the burner, be used to print out a correlation listing of the individual disc numbers 1 to 50, the alphanumeric character strings corresponding to each of the unique digital watermarks on the 50 discs, and the details of the recipient to whom each individual disc is then to be despatched, and if desired, a set of address labels for those recipients. The details may also be stored electronically in a suitable file format on a suitable storage medium, e.g. hard, disc, floppy disc, recorded optical disc.

If, following the pre-release distribution of the 50 discs, a pirate or counterfeit DVD is made from one of them, then, by looking at the digital watermark in the audio file on that disc, it can be determined which of the 50 pre-release DVDs was used as the master from which the counterfeit copies were derived, so enabling rapid and effective prosecution to take place. The present invention thus provides a further much-needed weapon in the armoury needed by the legitimate audio and audio visual creative and distribution industries in their fight against copyright infringement, counterfeiting and privacy.

Finally, although the present invention has been described in relation to watermarking techniques involving the insertion of notch filters, similar benefits occur using the invention with alternative watermarking techniques, such as for example spread-spectrum methods, which are within the scope of the present claims.

Claims

1. A method of watermarking an audio or audio visual work, the method comprising: - a. processing the audio or audio visual signal from said work a first time to provide a part-processed signal and auxiliary data for identifying characteristics for subsequent watermark insertion, and b. processing said part processed signal a further time to insert watermark data therein, the watermark data having characteristics determined by said auxiliary data.

2. A method as claimed in claim 1 further comprising: - c. recording the watermarked audio or audio-visual signal onto a data carrier to produce a watermarked recording.

3. A method according to claim 2 wherein anti-copying protection measures are also applied to the recording.

4. A method as claimed in any preceding claim in which the auxiliary data comprise information on when to insert the watermark data.

5. A method as claimed in any preceding claim in which the auxiliary data include a code amplitude profile for the watermark data..

6. A method as claimed in any preceding claim in which the auxiliary data is strongly attached to the part- processed signal.

7. A method as claimed in any preceding claim in which the part-processed data is stored as a .WAV file, and the auxiliary data is strongly attached thereto as helper data.

8. A method as claimed in any preceding claim in which step b . is performed a plurality of times to produce a plurality of copies of the same audio or audio-visual signal, each having different watermark data therein.

9. A method according to claim 8 including producing a data correlation between the watermark on each recording and a datafile identifying the particular recording or a party to whom it is to be distributed.

10. Apparatus for carrying out the method according to any preceding claim, and comprising, connected together for interaction adapted to carry out the method, a computer, means for recording data from a master recording, means for selecting audio data therefrom, means for applying a watermark to the selected data, and means for producing a recording faithful to the master recording and incorporating the watermark therein on an individual data carrier.

11. Apparatus as claimed in claim 10 further comprising means (other than by the watermark) to distinguish said data carrier uniquely from other data carriers on to which the same master recording has been applied.

12. Apparatus according to claim 10 or claim 11 wherein the means for producing a recording includes a CD or DVD burner.

13. Apparatus according to claim 10 or claim 11 and arranged to produce a datafile correlating the watermarks and the individual data carriers .