New M4 batch - U-534 P1030680

Message boards : News : New M4 batch - U-534 P1030680

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
matajan

Send message
Joined: 19 Apr 10
Posts: 6
Credit: 851,215
RAC: 0
Message 2514 - Posted: 13 Apr 2013, 17:05:26 UTC - in response to Message 2513.  

In my last post I explained one of the reasons:

"According to my tests, an IC of 0,044 IS A GOOD IC. The current algorithm despises this (LOW) IC and try to find the highest IC possible."

Another reason is the search for the greatest score value using trigrams. For example, the triagram ein has the highest score value and the algorithm search the highest sum of triagrams score value so that the ideal solution for it, would be always eineinein ciphertext, ignoring any other solution.

It is a complex and delicate problem, this text is good or just garbage? so I think we need a smarter algorithm.


Best regards, matajan.
ID: 2514 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2515 - Posted: 13 Apr 2013, 17:51:02 UTC - in response to Message 2514.  

I think the algorithm is pretty good considering the fact that it's used on a very short text. It also gets stronger as the message length increases.

With strong dictionary it easily breaks 72 letter ciphertexts. The problem is the fact that we don't know if a dictionary is good until the message is broken.

Also, I doubt you actually could get a text like 'einseinseins...' or something like that. For a random output, if the score increases on one part, the stecker gets messed up badly and the score of the rest of the message drops by a lot.
Surely there are 'local maximums' where the score goes up far beyond the average on a random decrypt (as can be seen on main page - top results of 1,9M where the average is just 1,6), however if the trigram dict fits the text, the plaintext score will be even higher.


M4 Project homepage
M4 Project wiki
ID: 2515 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
matajan

Send message
Joined: 19 Apr 10
Posts: 6
Credit: 851,215
RAC: 0
Message 2516 - Posted: 14 Apr 2013, 18:09:26 UTC - in response to Message 2515.  

Obtain a text as: "eineinein..." it is practically impossible. I meant a possible text formed with trigrams with a high value as garbage like "ein...von...die...ich..."

I just take a look at dict files, why exists a lot of entries with zero value? This only increases the processing time and is totally useless.


Best regards, matajan.
ID: 2516 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2517 - Posted: 14 Apr 2013, 18:33:42 UTC - in response to Message 2516.  

I think you are wrong.
The bigram/trigram arrays have constant size of [26][26] and [26][26][26], so there is no performance loss other than reading more data from file.

M4 Project homepage
M4 Project wiki
ID: 2517 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
matajan

Send message
Joined: 19 Apr 10
Posts: 6
Credit: 851,215
RAC: 0
Message 2518 - Posted: 14 Apr 2013, 19:35:23 UTC - in response to Message 2517.  

Sorry, you are right. I am seeing that are obligatory all combinations of 2 and 3 characters because the function implementation used all the combinations as index although its value is zero.

I think that optimizing these sum functions could increase the processing speed. What do you think about this?
ID: 2518 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2527 - Posted: 9 May 2013, 9:09:51 UTC - in response to Message 2518.  
Last modified: 10 May 2013, 8:35:43 UTC

Well, recently I've been testing highly optimized app for modern CPUs (beats any earlier versions), perhaps it will be released as default via plan_class.

The project status for today is:

wu_filename - dict - avg/min walks:

vroln72_1 - naval - 37/33
vroln72_3 - u534 - 60/54
fnygm76_1 - naval - 21/20
fnygm76_2 - u534 - 20/19

The processing rate (on average) is near 4k workunits/hour, which results in over 2 full walks over keyspace/day.
I'll post top results later today.

http://www.enigmaathome.net/static/3356xwn1mw/vroln721.txt
http://www.enigmaathome.net/static/3357x13h75/vroln723.txt
http://www.enigmaathome.net/static/3354x91s9m/fnygm761.txt
http://www.enigmaathome.net/static/3355xwtqe6/fnygm762.txt
M4 Project homepage
M4 Project wiki
ID: 2527 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2538 - Posted: 20 Jun 2013, 5:46:58 UTC - in response to Message 2527.  

I think that the 76 letters long FNYG MXHU is broken.

m4_fnygm76_2: score: 1902866 settings: C:FB:ZPEU:G528:CPDGEJFIKTLZMSNOQURW

leitungvvvuuustuetzpktxwwwhavenxxfffttteunszwozwovierhuermitvrrhhhvvvgeloest





M4 Project homepage
M4 Project wiki
ID: 2538 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
TBREAKER
Avatar

Send message
Joined: 26 Sep 11
Posts: 29
Credit: 0
RAC: 0
Message 2539 - Posted: 20 Jun 2013, 7:37:32 UTC - in response to Message 2538.  
Last modified: 20 Jun 2013, 7:47:09 UTC

CONGRATULATIONS!!!

You made it. A new world record, I guess. 76 letters!

AN LEITUNG VON U BOOT STUETZPUNKT WILHELMSHAVEN: FUNKTELEGRAMM EINS ZWO ZWO VIER HIER MIT RHV GELOEST.

Translation is:

To administration from U Boot Command (Headquater) Wilhelmshaven. Radio message 1224 solved with RHV (Hand Cipher System)...

GREAT!!!
-=> Breaking German Navy Ciphers - The U534 Enigma messages <=-
ID: 2539 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2541 - Posted: 20 Jun 2013, 8:11:05 UTC - in response to Message 2539.  

Thanks !

It was the 48th walk over the full keyspace, so lots of CPU time went into this.
Thanks for everyone who participated!




M4 Project homepage
M4 Project wiki
ID: 2541 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
TBREAKER
Avatar

Send message
Joined: 26 Sep 11
Posts: 29
Credit: 0
RAC: 0
Message 2542 - Posted: 20 Jun 2013, 8:25:05 UTC - in response to Message 2541.  
Last modified: 20 Jun 2013, 8:35:42 UTC

Ok it is message P1030713. I thought it was 680 ;-)
-=> Breaking German Navy Ciphers - The U534 Enigma messages <=-
ID: 2542 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2543 - Posted: 20 Jun 2013, 8:31:57 UTC - in response to Message 2542.  
Last modified: 20 Jun 2013, 8:37:52 UTC

The message I posted above is actually the P1030713.

P1030680 and P1030713 have been running in parallel due to limitations of the server software - there is no way to abort all work that was already sent to the clients.
Having a 110-120k work units in progress at any time, which is close to 4 full keyspace walks, hurts a bit when the message is broken and suddenly they're not needed anymore. Running two text instead of one reduces the 'wasted' CPU time by 50%.
M4 Project homepage
M4 Project wiki
ID: 2543 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile krzyszp

Send message
Joined: 2 Sep 07
Posts: 1
Credit: 1,952,259
RAC: 0
Message 2545 - Posted: 20 Jun 2013, 9:38:25 UTC - in response to Message 2543.  
Last modified: 20 Jun 2013, 9:38:34 UTC

CONGRATULATIONS!!!
Krzysztof 'krzyszp' Piszczek
Please check my Ebook Converter
ID: 2545 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2580 - Posted: 22 Jun 2013, 12:57:32 UTC - in response to Message 2545.  
Last modified: 22 Jun 2013, 12:58:14 UTC

A status update on the P1030680:

I paused the m4_vroln72_1 batch, which uses old "naval" dictionary, because I have a feeling it might not work too well. It fails on ~40% of U534 messages. Very few are completely unbreakable, but there are plenty of random 'decrypts' scoring higher than the plaintext.

Currently the server sends out only m4_vroln72_3 based on 'u534' dictionary set. I think I'll add a second batch which will use updated dictionary, I'm not sure yet if it will replace the current m4_vroln72_3 or if I'll let it run in parallel.

The recent break brought more participants and currently the average processing rate is a bit over 2 full walks/day, which is very impressive.


M4 Project homepage
M4 Project wiki
ID: 2580 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rilian
Avatar

Send message
Joined: 30 Jan 09
Posts: 5
Credit: 224,941
RAC: 0
Message 2582 - Posted: 22 Jun 2013, 20:39:42 UTC - in response to Message 2580.  

TJM, is it right that just only one message is remaining unbroken on Enigma@Home ?

http://enigma.hoerenberg.com/index.php?cat=Unbroken&page=P1030713
ID: 2582 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2583 - Posted: 22 Jun 2013, 20:45:37 UTC - in response to Message 2582.  

Yep, we have just one M4 message left, at least right now.

M4 Project homepage
M4 Project wiki
ID: 2583 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 2587 - Posted: 26 Jun 2013, 12:29:31 UTC - in response to Message 2583.  
Last modified: 26 Jun 2013, 12:30:13 UTC

The new batch m4_vroln72_4 is assigned to r1u534 dictionary set, which is a revised version of the previous u534 set. Not much has changed, but on a few tests I ran, the new revision seems to produce better results.
M4 Project homepage
M4 Project wiki
ID: 2587 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
CoolerMaster

Send message
Joined: 30 May 13
Posts: 1
Credit: 30,525
RAC: 0
Message 2592 - Posted: 29 Jun 2013, 14:24:20 UTC

I signed up because of the main text that described how I could help break the remaining Enigma messages. But after several weeks I've discovered that they have all been broken and now the project is taking on many other tasks not related to the original goal. I didn't sign up for that and am very disappointed that the main page no longer reflects the current status of the project, that's misleading. Therefore I'm discontinuing my help.
ID: 2592 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
dedo

Send message
Joined: 14 Sep 13
Posts: 2
Credit: 210,593
RAC: 0
Message 2664 - Posted: 14 Sep 2013, 18:35:59 UTC - in response to Message 2592.  

I signed up because of the main text that described how I could help break the remaining Enigma messages. But after several weeks I've discovered that they have all been broken and now the project is taking on many other tasks not related to the original goal. I didn't sign up for that and am very disappointed that the main page no longer reflects the current status of the project, that's misleading. Therefore I'm discontinuing my help.


I have signed up recently too, very nice project, but I'd like to know which messages have been broken and what we are working on right now. Is there a page somewhere that shows this information?
ID: 2664 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
dedo

Send message
Joined: 14 Sep 13
Posts: 2
Credit: 210,593
RAC: 0
Message 2665 - Posted: 14 Sep 2013, 20:41:41 UTC - in response to Message 2664.  

It looks like we are working on this right now. http://enigma.hoerenberg.com/index.php?cat=Unbroken&page=P1030680
ID: 2665 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Thomas Hansard

Send message
Joined: 23 Nov 13
Posts: 6
Credit: 595,836
RAC: 0
Message 2737 - Posted: 25 Nov 2013, 7:26:36 UTC

Is the current subject of this project P1030680?
ID: 2737 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : News : New M4 batch - U-534 P1030680




Copyright © 2024 TJM