method question!

Message boards : Number crunching : method question!

To post messages, you must log in.

AuthorMessage
Profile Tommy

Send message
Joined: 16 Sep 07
Posts: 7
Credit: 107,464
RAC: 0
Message 142 - Posted: 14 Nov 2007, 13:51:59 UTC

I wanted to know as the program tries the solution of the enigma message. The project uses a brute force method, trying all the possible combinations? As an example, what indicates the score in the solutions? Thanks.

ID: 142 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 143 - Posted: 14 Nov 2007, 13:53:58 UTC - in response to Message 142.  
Last modified: 20 Nov 2007, 0:11:41 UTC

The enigma keyspace is way too large for a brute force. Stefan Krah's software uses hillclimbing algorithms to find plugboard setting, which produces best result in a range of rotor/start position settings.
Full range of enigma settings is divided into smaller ranges, for alpha server it's 13 right ring settings and 26^4 start positions.

With current speed, it takes about 18 hours to do a full walk through all enigma settings*. However, single walk may not be enough to break a message. Some encryptions are harder to break, it has been proven, that sometimes hundreds or thousands walks are needed to break a tough message. A quote from Stefan's site:


Suppose that you have lost a gold coin in an area of one square kilometer. Fortunately, you have a metal detector. Unfortunately, the detector is such that you only have a 50% chance of getting an alert even if you hold it right above the coin.

You come up with a plan. Since you are a methodical person, you divide the square kilometer into 1000000 work units of one square meter each. Subsequently, you go over the whole area, processing each unit once. In 50% of the cases, you'll have found the coin. If you haven't, you'll go over the area a second time, unit by unit. Now your odds of having found the coin are 75%. And so on.

It should be clear by now that the square kilometer stands for the whole search space, which is divided up into 11388 work units. In the above statistic, we have searched each work unit about 60 times.

It should also be clear that the large number of remaining work units results from the fact that a maximum of 500 passes has been planned per work unit and the server reports the total number of units as 11388 * 500 = 5694000.


* - it's not entirely true, because due to some minor problems with the work wrapper and the way how BOINC network works, work units are not equally distributed.

Here you can read why bruteforce is useless.
M4 Project homepage
M4 Project wiki
ID: 143 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 144 - Posted: 14 Nov 2007, 13:54:28 UTC - in response to Message 143.  

A score tells how well the result resembles plaintext. The final score reported to the server is the best result found by your client within work unit range.
It's based on trigram count - if you take a look at 00trigr.naval, you'll find groups of three letters, each group has it score which depends on how often it appears in decrypted enigma messages.

If you want to know more about hillclimbing enigma messages, you can find interesting article here : http://frode.home.cern.ch/frode/crypto/bgac/index.html
M4 Project homepage
M4 Project wiki
ID: 144 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tommy

Send message
Joined: 16 Sep 07
Posts: 7
Credit: 107,464
RAC: 0
Message 269 - Posted: 22 Nov 2007, 18:42:03 UTC - in response to Message 144.  

A score tells how well the result resembles plaintext. The final score reported to the server is the best result found by your client within work unit range.
It's based on trigram count - if you take a look at 00trigr.naval, you'll find groups of three letters, each group has it score which depends on how often it appears in decrypted enigma messages.

If you want to know more about hillclimbing enigma messages, you can find interesting article here : http://frode.home.cern.ch/frode/crypto/bgac/index.html


Hi TJM, there is a mistake: this post was opened by me and not by Alfredo15. The first post was written by me (Tommy: p)!
Then i wanted to get a question ... the server status page... Hceyz72 work units ready: 5353552 ... the total number of wus? After the five million wus will certainly decrypted the message? For now we are about 350000 wus calculated, this means that we have analyzed the 15% of the total, right? Thanks!
ID: 269 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 270 - Posted: 22 Nov 2007, 19:08:13 UTC - in response to Message 269.  


Hi TJM, there is a mistake: this post was opened by me and not by Alfredo15. The first post was written by me (Tommy: p)!


That's because my script couldn't match the original post with any of users in the db (different email perhaps?). I'll fix it manually.


Then i wanted to get a question ... the server status page... Hceyz72 work units ready: 5353552 ... the total number of wus?


It's the number of hceyz72 workunits left.

After the five million wus will certainly decrypted the message? For now we are about 350000 wus calculated, this means that we have analyzed the 15% of the total, right? Thanks!


Take look at the number of keyspace walks. It's 30/500, less than 10%.
Either the message will be broken before reaching 100%, or we'll know that it's unbreakable with the current method.


M4 Project homepage
M4 Project wiki
ID: 270 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tommy

Send message
Joined: 16 Sep 07
Posts: 7
Credit: 107,464
RAC: 0
Message 355 - Posted: 14 Dec 2007, 9:51:26 UTC

I read that the maximum score achieved is 1883763. It is possible to know "a priori" (an estimate) what will be the score of a decrypted message of a certain length? You can make a statistical average of messages already decripted so that we can predict the score for messages not yet clear?
ID: 355 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 357 - Posted: 14 Dec 2007, 10:14:42 UTC - in response to Message 355.  
Last modified: 14 Dec 2007, 10:18:04 UTC

I read that the maximum score achieved is 1883763. It is possible to know "a priori" (an estimate) what will be the score of a decrypted message of a certain length? You can make a statistical average of messages already decripted so that we can predict the score for messages not yet clear?



Usually plaintext scores are 40-100% higher than highest 'garbage'. I'd expect score at least close to 2,6M for a full decrypt, but a lot depends on the message - while doing pre-BOINC tests I saw decrypts with scores only 10% higher than highest nonsense.

Examples:

http://www.bytereef.org/first-break-logs.txt - first break logs.

One of the challenge messages broken:


51084: -- B:AKZ:YEY:524:AECFGLHIKPMSNROU
ihykteinsseqsxaqtxviereinsxviergefallenexeinsnullverwuntetexneunzugang
xvierneunbestandxfeldlazaretteingesetztinxbhfxutorgoschxbhfxutorgoschx
hockxhockxsiegfriedsiegfriedxstandartenfuehrer

22030: -- B:AJY:UEK:125:ATBXCDFJGZHMKWLQ
mgqvjfichwekrjwkvvufkvnullpiqrlzmtxeinscoeiqgjugrobidgkfxgofwhnagbqwwh
irzknreinstattnjzdgecwgcwenmamhtvdbqwvbscabneegeinsohzcjvwueviebwgbtxh
nkbaijnxeintqkixofnsifueeblxgsznvhzmbencwsfbyb

21260: -- B:AFW:CEN:125:AYCVEJFOGWHQMUNS
lafbycspulleinsqabhexeinfmlehzppbtgasyhrureilcbrpersuhviabakzeditzmgnk
xgqenlgeinkxoamaqrzehtloledfrvormzvomxanngsueusglfabaplxvytrahubjgjaxr
poqmmnhbfndelovetjeinstxeinclhcbiqdihzhjublfdl

20965: -- B:AIL:JEA:124:ALBSCUDWFXHIKMOZ
kufqeinulqdnuugeinsdrfdimnulxjzjjyjbuprbgihvgelghcfaygoviaagxuezxkeine
insngovtyhnvcgcljbkbnwjthzlcttaaqnnqjpekurpvtjyenoxcskqfbhgkkhexkoupba
mnezyewkjcsgvndvwrlczeinullcdeayepdjixgeinolnp

20190: -- B:AIJ:AEC:123:ANCKGPHMLXOWQTSZ
gxisskewupxzmmxolehztvqdertabdyvketqtxbgoglwwxmphrrupzrozdeinsfoqtifca
nngrzyvqupirhazynviqfhgqpxeinssinqerehrpumezxderpngkjhcvlokiqfsdfoxisz
mugybnuqoeinergezzoahojegenttogveinszlzqaplfta

19775: -- B:ACX:WEF:123:ATCXDRFLGIHVJSKQ
zottdaipnajwnwteinsandeinsxxicogvnxargyiskjynisknuitcenwjcjhxwaeinddzp
qdtiflkenkrenmicjtujpksimseinhodvfwewqnrbibvausiwrngeuvqudegbwzokqevzj
zwjmiyopundzxtjfkbzfsvomuvyzdotebtcxfknquxeins


Trial decrypt of truncated message - top score only slightly higher than highest garbage. Also there's a partial decrypt below (score 15804 - stecker is correct, but ring and wheel settings are not).



19219: -- B:AIZ:XEH:524:AECFGLHIKPMSNROU
iereinsxaqtgefallenexdreisiebenklammereinsklammerv

17222: -- B:ALS:JEU:524:ABDYEKGPJVLOMTNZ
gosjjsqrweeeinskehhcxeeinseinseinmiruxfueinfnhcyax

16845: -- B:AOZ:ZEB:524:AVBOCPDREQFKGMHU
csxvcweinaueninsscqxxbpinsowbereinsinirvpurxeinsxa

15810: -- B:AAL:XEL:524:AJBKCDELFSGRHZIV
ktmdeinsazsejeijnbeinrgwbxeinosterllereinsxkawosbn

15804: -- B:AIY:XEG:524:AECFGLHIKPMSNROU
ieieinsxaqtgefallenexdreisiepenklammereinsklammerv

15405: -- B:AAN:LED:524:ASBCDKEIFLGYHNJQ
ljzyzeinssenxfleineagueqbereinaxkrspderwosseinkaqe

15260: -- B:AAO:ZEH:524:AZCPDJEQFIGXHTKV
ceinshanmanjjbqgxeinsnzqeinuscgzajeinrchegeindkykb

15179: -- B:AAW:PEU:524:AHCIDQFVGKJLMTNR
eeinssinsvztdaeincvggeinxkusiefgsbtogcrovuoxeinskv


M4 Project homepage
M4 Project wiki
ID: 357 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tommy

Send message
Joined: 16 Sep 07
Posts: 7
Credit: 107,464
RAC: 0
Message 366 - Posted: 16 Dec 2007, 12:15:57 UTC

What is the "criticality" of this message that we are trying to decrypt? It seems to me that this is harder to solve. Thanks
ID: 366 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 368 - Posted: 17 Dec 2007, 0:06:31 UTC - in response to Message 366.  

It's definitely a tough one - previous two were broken in just few weeks, this one took already lots of CPU time and it's still unbroken. Maybe its an 'offizier' (double encrypted) message, which probably couldn't be broken with current software, or it's too garbled or has too many missing letters to decrypt. We won't know until it's broken.

M4 Project homepage
M4 Project wiki
ID: 368 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Tommy

Send message
Joined: 16 Sep 07
Posts: 7
Credit: 107,464
RAC: 0
Message 380 - Posted: 23 Dec 2007, 9:54:35 UTC - in response to Message 368.  

It's definitely a tough one - previous two were broken in just few weeks, this one took already lots of CPU time and it's still unbroken. Maybe its an 'offizier' (double encrypted) message, which probably couldn't be broken with current software, or it's too garbled or has too many missing letters to decrypt. We won't know until it's broken.


In some cases, the Germans using double-encrypted messages (offizier). I read that transformed the message with a specific codebook and then crypted with enigma. So with the current client is impossible to decrypt a message double-encrypted :(. This case will have a particularly important content :).
ID: 380 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : method question!




Copyright © 2024 TJM