Automatic plagiarism detection in obfuscated text

Abstract

Plagiarism is a serious problem in education, research, publishing and other fields. Automatic plagiarism detection systems are crucial for ensuring the integrity and genuineness of intellectual work. There are different types of plagiarism, such as copy–paste, obfuscation and translation. In particular, obfuscated text is one of the hardest types of plagiarism to detect. In this paper, we propose an automatic plagiarism detection system for obfuscated text based on a support vector machine classifier that exploits a set of lexical, syntactic and semantic features. We evaluated the performance of the proposed system on benchmark English and Arabic corpora made available by the PAN Workshop series: PAN 2012, PAN 2013, PAN 2014 and PAN@FIRE2015. We also compared the performance of our system to the performances of other systems that participated in the PAN competitions. The obtained results show that our system had the best performance in terms of the F-measure on the PAN 2012 and on the PAN@FIRE2015 obfuscated sub-corpora, was among the top four on the PAN 2013 corpus and was among the top two on the PAN 2014 corpus.

Appendix: Output examples

This appendix contains, for each obfuscation type, some plagiarism cases from PAN 2012 through 2014 as reported by [51,52,53]. The highlighted texts are the text segments detected by our system.

Example 1: Simulated Plagiarism PAN12 source-document02500 suspicious-document02500.

Suspicious text: Arrest, if in Cincinnati, William Wood, a friend of Jackson. Cargo as an accomplice. About 20 years old, 5 feet 11 inches, blond hair, face taut, rather thin, weighs 165 pounds. We’re going from here to South Bend after Wood to leave here to there. CRIM McDermott and Plummer. ”

Immediately after receiving the telegram Col. Deitsch Witte detailed Detectives, Bulmer and Jackson to take care of Jackson. It was learned that stayed at the house of Mrs. McNevin, at 222 West Ninth, next to the Opera House Robinson.

Detective Jackson was stationed at home and Witte and Bulmer in front of the room. Just when it seemed he had found his game for the fact that the officers were after him and had gone to unknown places was captured. It was after nine o’clock, when most of the last ray of hope had died out of the breasts officers, the Chief of Police Deitsch received the news that Jackson had been seen at the Palace Hotel.

The chief began and I met a man answering description of Jackson. He informed the detectives of reality, the individual was last seen and was walking slowly along Ninth Street, and when he reached 222 he looked up at the windows. He walked slowly to Plum Street and stopped and looked back toward the house.

Then hurried north on Plum Street to the Court. When the route was part of the square Bulmer detective approached him, saying: “Your name is Jackson, right?” He turned completely pale and trembling like an aspen, and as the detective continued, “I love you” he exclaimed, “My God, what is this?

At the same time, there was the beginning of the Mayor’s Office

Source text: Arrest if in Cincinnati, William Wood, friend of Jackson. Charge as accomplice. About 20 years, 5 feet 11 inches, light blonde hair, smooth face, rather slender, weighs 165 pounds. We go from here to South Bend after Wood as he left here for that place. CRIM MCDERMOTT AND PLUMMER.”

Immediately on receipt of the telegram Colonel Deitsch detailed Detectives Witte, Bulmer and Jackson to look after Jackson. It was learned that he roomed at the house of Mrs. McNevin, at 222 West Ninth, next door to Robinson’s Opera House. Detective Jackson was stationed in the house and Witte and Bulmer in the saloon opposite.

Just when it seemed as though their intended game had discovered the fact that the officers were after him and had left for parts unknown he was captured.

It was after 9 o’clock, when almost the last ray of hope had died out of the officers breasts, that Chief of Police Deitsch received word that Jackson had just been seen at the Palace Hotel. The chief started out and ran into a man answering Jackson’s description. He informed the detectives of the fact, the fellow was watched and was seen to walk slowly down Ninth Street, and on reaching 222 he looked up at the windows. He strolled slowly to Plum Street and stopped and again looked back at the house.

He then walked rapidly north on Plum Street toward Court. When he had traversed part of the square Detective Bulmer stepped up to him, saying: “Your name is Jackson, isn’t it?” The man turned perfectly livid and trembled like an aspen, and as the detective continued to say, “I want you,” he exclaimed, “My God! what is this for?”

At the same time the start was made for the Mayor’s Office.

Example 2: High artificial obfuscation PAN12 source-document01505 suspicious-document01505.

Suspicious text: He bestir, when you say than there the message as the Cheops, that you should not shoot whether your Escape there is as visit as i establish mine to be.$*****$ Mister. Of christopher and CULLEY, whom he may never talk how i have from bustling one, ill cinematic message_ The noemi of an Elevation by$*$, has powerfully melt up_ atrocities to a, thus good. Latter Alleyway of(CASSELL) is, in malice of which i cannot excessively really reject, as successful part to anyone need want, come a writer to have been postdate of headway_ decoupage. Wondrous_ scenario is not prussian Direction, in circumference, and the tearjerker being_ fifty dudgeon by his prussian national by House, and what do he look yet the place at WA would again in emblem. The “Alleyway to” of prussian gens is own shantytown in Dhegiha$*$ War whether the_, stranded atrocities of mankind has really, has met for boy by the vicissitudes of dystopia were well be to articulation him do. I had to labialize that one of Mister. CULLEY’Karl trouble will have represented in case, when the message, leading love, had today to be bored at debris of all villagers, and have a association was shortly more urgent conversations. Violently of the betrayed and review the Kentan of is bring at-_, and she is lie, necessarily to retrieve, as short intentions are of potential miseries_ support, but had excessively cognize for_ Dave’mho_ in day for be hustle merely away of a idiom of a eventually final (and hastily be thrilling) souls. The account, that i has to believe, than the performer, whom he go prominently thus, is for—Mister. CULLEY year. The Dave$*$_,_ whom has been_ narrative, is not the juvenile which do he are demo entirely in another estimation events up its truthful quality; or on the subject_ Dave’_ may not have particularly rotate eventually be unimportant troubles of ground of a performer,_. For him are eventually i will have certainly visualize the volume before one not to be lose with one who lessens that texture at westward rugged or masterful weeks.$*****$_ Life Objects and Conversations in (Gracie and Edith) is there are least publication that he will be formerly see varied cer to which have been the who deplored it. Sorrow there can be of blade and cerebral panel who give; anglophilia on cowardice nor to tour that things make of the agency caused that the brother beside those it has been for the; and vehement with the troubles of another position (against the_ are o’er) keep after the message by the autobiography.

Source text: I hope, when you read this tale of the Pharaohs, that you will not find that your memory of the Book of Exodus is as faded as I found mine to be.

$* * * * *$

Mr. CHRISTOPHER CULLEY, whom you may remember for a bustling, rather cinematic story called_Naomi of the Mountains_, has now followed this with another, considerably better._Lily of the Alley_ (CASSELL) is, in spite of a title of which I cannot too strongly disapprove, as successful a piece of work of its own kind as anyone need wish for, showing the author to have made a notable advance in his art. Again the setting is Wild West, on the Mexican border, the theme of the tale being the outrages inflicted upon American citizens by VILLA, and what seemed then the bewildering delay of Washington over the vindication of the flag. The “Alley” of its unfortunate name is the slum in Kansas City where _Dave_, stranded on his way westward, met the girl to whom the laws of fiction were inevitably to join him. I fancy that one of Mr. CULLEY’S difficulties may have lain in the fact that, when the tale, following _Dave_, had finally shaken itself from the dust of cities, the need for feminine society was conspicuously less urgent. Even after a rescued and refreshed _Lily_ is brought up-country, she is kept, so to speak, as long as possible at the base, and only arrives on the actual scene of _Dave’s_ activities in time to be bustled hurriedly out of the way of the final (and wonderfully thrilling) chapters. The explanation is, I think, that the cowboy, whom he knows so well, is for Mr. CULLEY hero and heroine too. _Dave_, round whom the story revolves, is a pleasant study of a type of American youth which we are coming gratefully to estimate at its true worth; but in the development of the theme _Dave_ soon becomes almost insignificant beside the greater figure of the cowboy, _Monte Latarette_. For him alone I should regard the book as one not to be missed by anyone who values a handling of character at once delicate and masterful.

$* * * * *$

_Keeling Letters and Recollections_ (ALLEN AND UNWIN) is a book that will perhaps rouse varied emotions in those who read it. Regret there will be for so much youth and intellectual vigour sacrificed; admiration for courage and for a patriotism that circumstances made by no means the simple matter of conviction that it has been for most; and vehement opposition to many of the views (on the War especially) held by the subject of the memoir.

Example 3: Low artificial obfuscation PAN12 source-document01000 suspicious-document01000.

Suspicious text: The call of, “Pig away!” and the dash of bairn in the pursuit, at last make such a soprano that both attend grey and the much-try Andrew make disorder to the vicar. Time great declared that discipline was become impossible, and Andrew that there would not be a “martal vegetable in the Master Robinette’s’hog you got out thus merely.” Then have the vicar it made the concept for this consequence: “if David’s polly porker is seen in a duel had again, it travel stern that same time to Farmer.” The priest’s regulation were not flagship to be disregarded, and his threats were always carried away. David and Ambrose might have been understand with a cock and nails very busy avenues at the hog-hordeolum that afternoon, and Antony’mho visits to the garden discontinue, until one unlucky occasion when David was away from condominium alas, and it fell away in the mode:—in the town of Nearminster, ten knot from Easney, populate Pennie’s godparent Miss Unity Cheffins, and it was Mr and Mrs Hawthorn’grass to pay than her did an choice of 2 or 3 days, taking each kinds of the dean bairn with them in curve. It was an occasion much expect by latter, but more for the glory in the situation than from any delight being connected with it, for Miss and Unity was instead come a potent young begum, and particular strength in her notions as to their poor jealousy. She was fond of saying, “in what_ my_ time young fortune did so and often,” and of noticing any failure from all niceness, or yet any shortcoming. She was a proper young lady, and lived in a firm so outside the Cathedral close; it was sombrely furnished, and full of dark young portraits, and rare China bowling and knick-knacks, which last Woman Completeness thought a transaction of, and dusted carefully with a hands.

Source text: The cry of, “Pig out!” and the consequent rush of children in pursuit, at last reached such a pitch that both Miss Grey and the much-tried Andrew made complaint to the vicar. Miss Grey declared that discipline was becoming impossible, and Andrew that there would not be a “martal vegetable in the garden if Master David’s pig got out so often.” Then the vicar made a rule to this effect:

“If David’s pig is seen in the garden again, it goes back that same day to Farmer Hatchard.”

The vicar’s rules were not things to be disregarded, and his threats were always carried out. David and Ambrose might have been seen with a large hammer and nails very busy at the pig-sty that afternoon, and Antony’s visits to the garden ceased, until one unlucky occasion when David was away from home, and it fell out in the following manner:-

In the cathedral town of Nearminster, 10 miles from Easney, lived Pennie’s godmother Miss Unity Cheffins, and it was Mr and Mrs Hawthorn’s custom to pay her an annual visit of 2 or 3 days, taking each of the four elder children with them in turn. It was an occasion much anticipated by the latter, but more for the honour of the thing than from any actual pleasure connected with it, for Miss Unity was rather a stiff old lady, and particular in her notions as to their proper behaviour. She was fond of saying, “In_my_ time young people did so and so,” and of noticing any little failure in politeness, or even any personal defect. She was a rich old lady, and lived in a great square house just inside the Cathedral Close; it was sombrely furnished, and full of dark old portraits, and rare China bowls and knick-knacks, which last Miss Unity thought a great deal of, and dusted carefully with her own hands.

Example 4: Random artificial obfuscation PAN13 source-document00921 suspicious-document00086.

Suspicious text: When responding, please specify exactly which program you are previous in. rsvp only whfellow@ writing. upenn edu here whfellow@ writing. upenn Seminar, students will study the work Dog. This program. The edu is teach by Professor. Al.-Lee syllabi from age are available.

Source text: When responding, please specify exactly which program you are interested in.rsvp only whfellow@writing.upenn.edu here whfellow@writing.upenn.edu In the Writers House Fellows Seminar, students will study the work of all three Fellows. The course is taught by Kelly Professor and Writers House Faculty Director . This year’s coordinator of the program is . Al Filreis Jamie–Lee Josselyn The syllabi from previous years are available.

Suspicious text: We desire to make it possible for the youngest writers and abstractor-evaluator to have sustained contact with abstractor of accomplishment. We besides desire to resist the clip-honored differentiation—more honored in biologism between working with eminent writers on extremity and analyse literature on the other.

Source text: We want to make it possible for the youngest writers and writer-critics to have sustained contact with authors of great accomplishment in an informal atmosphere. We also want to resist the time-honored distinction—more honored in practice than in theory—between working with eminent writers on the one hand and studying literature on the other.