A GPU version of Enigma software

Message boards : Number crunching : A GPU version of Enigma software

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 11 · Next

AuthorMessage
Jeremy

Send message
Joined: 14 Dec 12
Posts: 1
Credit: 8,563,480
RAC: 23,590
Message 4619 - Posted: 7 May 2017, 20:51:00 UTC

Spent: 00:00:16
Pass: 1
Score: 83067
Words: -1008
Key: B:524:KZ:YED
Plugs: AECFGLHIKPMSNROUQYTW
Text: EINS SEQS X AQT X VIER EINS X VIER GEFALLEN e X EINS NULL verwuntete X NEUN ZUGANG X VIER NEUN BESTAND X FELD LAZARETT EINGESETZT in X bhf X u TOR gosch X bhf X u TOR gosch X hock X hock X SIEGFRIED SIEGFRIED X STANDARTENFUEHRER

DONE: 1 passes in 00:00:38

the keys and plugs are the same

Key: B:524:KZ:YED
Plugs: AECFGLHIKPMSNROUQYTW

Key: B:524:KZ:YED
Plugs: AECFGLHIKPMSNROUQYTW
NVIDIA GeForce GTX 750 Ti
Manufacturer NVIDIA
Model GeForce GTX 750 Ti
Device ID 10DE-1380
Revision A3
Subvendor EVGA (3842)
Current Performance Level Level 0
Current GPU Clock 135 MHz
Current Memory Clock 405 MHz
Current Shader Clock 405 MHz
Voltage 0.937 V
Current Performance Level Level 0
Current GPU Clock 135 MHz
Current Memory Clock 405 MHz
Current Shader Clock 405 MHz
Voltage 0.937 V
Technology 28 nm
Bus Interface PCI Express x16
Temperature 31 °C
SLI Disabled
Driver version 21.21.13.7653
BIOS Version 82.07.55.00.50
Physical Memory 2047 MB
Virtual Memory 2048 MB
ID: 4619 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 4620 - Posted: 8 May 2017, 21:05:56 UTC - in response to Message 4619.  
Last modified: 8 May 2017, 23:41:54 UTC

The server is almost ready to support the new app. For testing I added only the 32bit win version, but once the backend is finished it will be easy to add other platforms.
I'll add a couple of test tasks today just to check if the work generator is working and the server can receive results.

EDIT: everything seems to be working (app versions below 1.04 are broken, also some of the released workunits are also broken). The message I used for test (72 letters M4, search on known wheel order with unknown rings, key and stecker) is solved already, even though just a few results were returned.
I'm going to work on validator today.
M4 Project homepage
M4 Project wiki
ID: 4620 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BlackObelisk

Send message
Joined: 30 May 16
Posts: 2
Credit: 8,024,484
RAC: 4,330
Message 4622 - Posted: 9 May 2017, 17:43:41 UTC

gtx 1050ti 20 sec
ID: 4622 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 4844 - Posted: 14 May 2017, 20:51:50 UTC

Since this version is not compatible with standard app, it could be added as a third app, with different keyspace/workunit distribution system (the app that runs uniscore as a first pass shares work generator with 'old' app).

What's the proper command line to run a test against M4 message ? I'd like to see how fast it is because app speed determines min/max workunit key range.


M4 Project homepage
M4 Project wiki
ID: 4844 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Alex VE3NEA

Send message
Joined: 22 Oct 16
Posts: 36
Credit: 128,745
RAC: 0
Message 4861 - Posted: 15 May 2017, 23:25:31 UTC

I have uploaded a sample run_m4.bat file with one possible set of command line parameters, along with an M4 test message and some dictionaries (that do not work well on this message, please use your own ones). The exe files have also been updated, I have fixed a bug related to upper/lower case letters in the M4 keys, please re-download them as well.

Here is a sample command line:

enigma-cuda.exe -M M4 -f c:b568:AA:AAAA -t c:b568:ZZ:AZZZ -a -g 013 00trigr.cur 00unigr.cur P1030667.txt

Here -a means try the keys with and without a slow wheel turnover, -g 013 means use unigrams. (wthout -g 013 it would use bigrams). Unigrams result in somewhat higher success rate than bigrams.

The thin reflectors and Greek wheels are supposed to be lowercase, but with the bug fixed, upper case also works.

Note that for performance reasons, the program iterates over the key parts in this order:

1) left-middle-right rotor positions;
2) ring positions;
3) Greek wheel position;
4) rotor order;
5) reflector.

-- so the following blocks are possible (X is a fixed letter):

-f c:b568:XX:XAAA -t c:b568:XX:XZZZ
-f c:b568:XA:XAAA -t c:b568:XZ:XZZZ
-f c:b568:AA:XAAA -t c:b568:ZZ:XZZZ
-f c:b568:AA:AAAA -t c:b568:ZZ:ZZZZ

On GTX-1070, these blocks are processed in 0.3 s, 4 s, 52 s and 1345 s respectively. On the low end video cards the speed might be much slower, here is my current comparison table (PBNXA message):
video card            time, s
-------------------------------
GTX 1070                    8
GTX 1060                   10.5
GTX 980 Ti                 10
GTX 1050 Ti                20
GTX TITAN Black            27
GTX 750 Ti                 38
GTX 760 (50% overclock)    54
GTX 660m                  218
GT  420                  1280




ID: 4861 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 4895 - Posted: 18 May 2017, 8:51:40 UTC
Last modified: 18 May 2017, 8:57:29 UTC

Unigrams result in somewhat higher success rate than bigrams.


Actually it's much higher:


run #	bigram    unigram
1       66        147
2       13        66
3       245       140
4       68        68
5       53        143
6       599       52
7       17        66
8       587       80
9       1266      4
10      131       100
11	196	58
12	82	134
13	75	16
14	383	120
15	370	25
16	379	86
		
average	283	81



81 vs 283 average number of passess. I was using unigr.u534 and bigr.u534 both created from the same sample.
M4 Project homepage
M4 Project wiki
ID: 4895 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergey Kovalchuk

Send message
Joined: 29 Oct 15
Posts: 2
Credit: 111,161
RAC: 0
Message 4936 - Posted: 24 May 2017, 12:08:46 UTC - in response to Message 4844.  

Since this version is not compatible with standard app, it could be added as a third app, with different keyspace/workunit distribution system (the app that runs uniscore as a first pass shares work generator with 'old' app).


At least approximately when you can expect the appearance of the GPU version of the application
ID: 4936 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Alex VE3NEA

Send message
Joined: 22 Oct 16
Posts: 36
Credit: 128,745
RAC: 0
Message 4956 - Posted: 27 May 2017, 19:15:59 UTC

The source code of the GPU-based Enigma, enigma-cuda, is available here:

https://github.com/VE3NEA/enigma-cuda
ID: 4956 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5141 - Posted: 5 Jun 2017, 22:23:11 UTC - in response to Message 4936.  


At least approximately when you can expect the appearance of the GPU version of the application


It will take a while to build fully BOINC-compatible app with integrated API, but as soon as the server is ready, perhaps this weekend or early next week, it should be possible to run "standalone" app via wrapper.
It will probably be marked as beta for now and disabled in preferences by default, because without the BOINC API there are some problems and glitches.

I have the batch->workunits splitter ready, it supports two of the largest block sizes but for the <100 letters M4 text it does not make sense to run anything shorter than the largest one (AA:AAAA->ZZ:ZZZZ) as the server probably would not have enough resources to keep up with workunit and data processing. On GTX 1050 it takes 3 to 7 minutes to process a WU of that size; the time changes with ring settings.
As soon as I get the validator working I'll be releasing test workunits.
M4 Project homepage
M4 Project wiki
ID: 5141 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Alex VE3NEA

Send message
Joined: 22 Oct 16
Posts: 36
Credit: 128,745
RAC: 0
Message 5175 - Posted: 9 Jun 2017, 17:07:07 UTC

enigma-cuda.exe has just cracked another previously unbroken message from CryptoCellar, SIPVX.

Key: B:521:MRP:HEN
Plugboard: AGBJCPDSERFQHVIUKTLW
Text: Bitte um Angabe des Farsqqeges x Befinde miq in x Rosenow Rosenow x Sofort qunkqntwort x Waschbusch Wischbusch x
ID: 5175 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
oh2hyt

Send message
Joined: 14 Jul 09
Posts: 52
Credit: 180,944,794
RAC: 981,915
Message 5186 - Posted: 13 Jun 2017, 23:10:48 UTC

@TJM
Could you share app_info.xml details for enigma_cuda once you have done internal testing, please. You seems to be progressing nicely.
ID: 5186 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5190 - Posted: 14 Jun 2017, 15:08:26 UTC - in response to Message 5186.  
Last modified: 14 Jun 2017, 15:10:51 UTC

I finally got a few of test workunits validated. For today I have planned the last 'closed' test batch and then I'll delete all the GPU workunits to release first public test.

To release the test app I have to rebuild the wrapper - by default it adds "--device x" to the command line to tell the app which CUDA device to use, but the app does not understand that and returns an error.
Even better option would be to patch application to support "--device x" passed from command line, as it would allow running on selected device (or multiple devices at once) on systems with more than one GPU, but for that we have to ask the author as at this moment I don't have cuda environment installed.

For testing purposes I omitted "coproc" section in app_info so the client thinks that it is a CPU app, it runs fine except that the client starts up multiple tasks at once.
M4 Project homepage
M4 Project wiki
ID: 5190 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
forretrio
Avatar

Send message
Joined: 25 Dec 14
Posts: 9
Credit: 3,283,768
RAC: 1
Message 5194 - Posted: 15 Jun 2017, 7:52:55 UTC

g4_vroln72_12 showed that there are ~20k units remaining, but I see 0 in the task by application table.

Is that intentional?
ID: 5194 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5198 - Posted: 15 Jun 2017, 20:50:17 UTC - in response to Message 5194.  

It was a test batch and it is disabled already.
At this moment GPU tasks availability is very limited until all major problems are fixed.
M4 Project homepage
M4 Project wiki
ID: 5198 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
oh2hyt

Send message
Joined: 14 Jul 09
Posts: 52
Credit: 180,944,794
RAC: 981,915
Message 5199 - Posted: 15 Jun 2017, 21:02:33 UTC - in response to Message 5198.  

How we should get those cuda tasks to compute? Even there tasks available, client won't download those. I assume because of there is no public app for cuda (listed in applications page)? So all who have got those currently have some special app_info.xml?
ID: 5199 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5252 - Posted: 17 Jun 2017, 8:59:08 UTC - in response to Message 5199.  
Last modified: 17 Jun 2017, 9:00:40 UTC

It used app_info + tweaked client configuration to run. There was only one tester other than me.

Later today I'll post the app_info that requires no mods on the client side, combined with a first batch of test workunits. This will be the first batch with credits set somewhere near the production workunits. Until now all the GPU test workunits had credits set to 1 or 2 per task, just to see if validator works.
M4 Project homepage
M4 Project wiki
ID: 5252 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5254 - Posted: 17 Jun 2017, 22:00:45 UTC
Last modified: 20 Jun 2017, 21:01:34 UTC

Here is an app_info.xml (just the xml) suitable for running all 3 apps. Please note that it is an early release which is not even beta yet. Do not use unless you know what you're doing.

https://s3.eu-central-1.amazonaws.com/enigma4/beta_gpu_apps/app_info.zip

You have to supply the wrappers and app executables and either rename them to names used in XML file, or edit the XML.

CPU app wrapper:
http://download.enigmaathome.net/wrapper_5.32_windows_intelx86.exe
enigma av x86
http://download.enigmaathome.net/enigma3_1.05_windows_intelx86.exe
enigma av x76-64
http://download.enigmaathome.net/enigma3_1.05_windows_x86_64.exe
enigma 0.76
http://download.enigmaathome.net/enigma_5.32_windows_intelx86.exe

job files:
https://s3.eu-central-1.amazonaws.com/enigma4/beta_gpu_apps/job_files.zip

The wrapper for GPU app is here:
https://s3.eu-central-1.amazonaws.com/enigma4/beta_gpu_apps/wrappergpu_1.05_windows_intelx86__cuda.zip
It's x86 but it does not matter, it'll run either the 32 or 64bit cuda app.
Latest GPU app executable is here: https://github.com/VE3NEA/enigma-cuda
rename it to enigmacuda_1.04_windows_intelx86__cuda.exe or edit the app_info (note: it says "intelx86" but it does not matter for anonymous platform, just the filename and app_info entry must match)

To get GPU tasks you have to edit your project prefs to allow beta apps. Enabling that option without app_info will result in download errors as there are no files on the server yet.

Please note that the app runs via wrapper and according to BOINC wiki, suspending or stopping/cancelling GPU app while it is running the kernel, may cause problems https://boinc.berkeley.edu/trac/wiki/AppCoprocessor see the section "Do GPU kernels within critical sections".
Also, in it's current state the app will run only on device 0.

EDIT#1: Forgot about the job files, link added.
EDIT#2: Fixed missing link to app_info
M4 Project homepage
M4 Project wiki
ID: 5254 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
oh2hyt

Send message
Joined: 14 Jul 09
Posts: 52
Credit: 180,944,794
RAC: 981,915
Message 5255 - Posted: 17 Jun 2017, 22:44:30 UTC - in response to Message 5254.  

Thanks.
Installed with few modifications CPU applications' side. Now waiting workunits :)

You supplied 1.05 Enigma AV executables, 1.04 Enigma AV's job file (they are identical), and app_info is using 1.04 Enigma AV filenames. So it won't work straight.
ID: 5255 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
oh2hyt

Send message
Joined: 14 Jul 09
Posts: 52
Credit: 180,944,794
RAC: 981,915
Message 5257 - Posted: 17 Jun 2017, 23:01:11 UTC

Managed to get one task for gpu. Went fine, but runtime was surprisingly short, just about 3 and half minutes.

Server seem to prefer sending cpu tasks over gpu tasks, so it just about luck to get something for gpu. I can't force project to prefer gpu in project settings, because then other computers won't get cpu tasks at all (other computers just error to no enigma_cuda available).
ID: 5257 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 68,346,081
RAC: 369,774
Message 5258 - Posted: 17 Jun 2017, 23:09:43 UTC - in response to Message 5257.  

This is not even a beta version yet, so do not expect large amount of tasks.
Also, you can use host venue to add separate prefs for single host.
M4 Project homepage
M4 Project wiki
ID: 5258 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 11 · Next

Message boards : Number crunching : A GPU version of Enigma software




Copyright © 2017 TJM