High Performance Computing

[(Sandman) Logan-5]

The new Quadro's are scheduled to be released this October. (Raytracing and rendering)
Check out these specs:  http://www.digitaleng.news/de/editors-pick-nvidia-quadro-gpus/

More details:  http://www.anandtech.com/show/10516/nvidia-announces-quadro-pascal-family-quadro-p6000-p5000

The new cards now incorporate 16 and 24 GB ECC memory. 

Projected pricing: $2,000.00 - $5,000.00 each.

[(Sandman) Logan-5]

Fast GPU rendering, not precise, but quick.

http://furryball.aaa-studio.eu/aboutFurryBall/index.html

CUDA OptiX based renderer.


Features:  http://furryball.aaa-studio.eu/aboutFurryBall/features.html

[(Sandman) Logan-5]

AMD VS NVIDIA for video editing:

http://www.gpurendering.com/technology/CudaVsOpencl.html

[(Sandman) Logan-5]

MFM: Kind of a bulky way of doing it (info from 2009), but to just slap some machines together that you have laying around, it's a pretty good article.
http://www.techradar.com/news/computing/pc/how-to-build-a-powerful-distributed-computer-515197

Pre-calculations before building: http://hpl-calculator.sourceforge.net/
and
http://www.wikihow.com/Build-a-Supercomputer

IPMI is another reason to use a SuperMicro MoBo for the Head Node, it's baked right into an embedded chip. There have been some hellacious exploits against IPMI, so make sure you look into that and run a firewall between your public and private nets if you'll be running your build with an internet connection. If it's a single purpose machine, and you're the only one with physical access and no internet connection, then you shouldn't have to worry. IPMI makes administrating the nodes very simple.

https://community.rapid7.com/community/metasploit/blog/2013/07/02/a-penetration-testers-guide-to-ipmi

http://www.tripwire.com/state-of-security/vulnerability-management/supermicro-ipmibmc-vulnerability-analysis/

https://isc.sans.edu/forums/diary/IPMI+Hacking+servers+that+are+turned+off/13399

Mitigation:



[url=https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf

https://www.microway.com/knowledge-center-articles/ipmi-security-for-datacenters-and-compute-clusters/

https://www.reddit.com/r/homelab/comments/4uun4y/note_to_self_never_build_another_server_without/

https://www.servethehome.com/download-supermicro-ipmiview-latest-version/

https://www.gnu.org/software/freeipmi/freeipmi-faq.html

http://www.identityweek.com/running-ipmi-lights-out-management/

Your best bet, use non-IPMI boards, or don't connect to the net - just use the cluster as a dedicated machine.

[Walks_At_Night]

Last high end system I worked on when I was in Server development:

x3950 X6

[albrecht]

MFM: Kind of a bulky way of doing it (info from 2009), but to just slap some machines together that you have laying around, it's a pretty good article.
http://www.techradar.com/news/computing/pc/how-to-build-a-powerful-distributed-computer-515197

Pre-calculations before building: http://hpl-calculator.sourceforge.net/
and
http://www.wikihow.com/Build-a-Supercomputer

IPMI is another reason to use a SuperMicro MoBo for the Head Node, it's baked right into an embedded chip. There have been some hellacious exploits against IPMI, so make sure you look into that and run a firewall between your public and private nets if you'll be running your build with an internet connection. If it's a single purpose machine, and you're the only one with physical access and no internet connection, then you shouldn't have to worry. IPMI makes administrating the nodes very simple.

https://community.rapid7.com/community/metasploit/blog/2013/07/02/a-penetration-testers-guide-to-ipmi

http://www.tripwire.com/state-of-security/vulnerability-management/supermicro-ipmibmc-vulnerability-analysis/

https://isc.sans.edu/forums/diary/IPMI+Hacking+servers+that+are+turned+off/13399

Mitigation:



[url=https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url=https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url=https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf]http://www.fail2ban.org/wiki/index.php/Main_Page['url]

[url]https://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf

https://www.microway.com/knowledge-center-articles/ipmi-security-for-datacenters-and-compute-clusters/

https://www.reddit.com/r/homelab/comments/4uun4y/note_to_self_never_build_another_server_without/

https://www.servethehome.com/download-supermicro-ipmiview-latest-version/

https://www.gnu.org/software/freeipmi/freeipmi-faq.html

http://www.identityweek.com/running-ipmi-lights-out-management/

Your best bet, use non-IPMI boards, or don't connect to the net - just use the cluster as a dedicated machine.

This stuff is beyond me but all I know is when my ship comes in and I got the big bucks I'm hiring (Sandman) Logan-5 to set up my computer system and smartphones. 

[(Sandman) Logan-5]

Last high end system I worked on when I was in Server development:

x3950 X6

Sweet.

https://community.spiceworks.com/topic/1675234-new-it-products-free-tools-cloud-based-services-and-security

[(Sandman) Logan-5]

Hardware accelerated core-to-core chip communication. Interesting. http://phys.org/news/2016-09-hardware-core-to-core-on-chip.html

[Mind Flayer Monk]

Thank you Sandman.

That 3D digitizer you posted in the other thread was interesting too. I don't know how much they cost-probably $70,000.

[(Sandman) Logan-5]

If you've gone the NVidia route, here a couple of really good articles loaded with resources.

https://devblogs.nvidia.com/parallelforall/how-build-gpu-accelerated-research-cluster/

https://devblogs.nvidia.com/parallelforall/12-things-tesla-accelerated-computing-platform/

[(Sandman) Logan-5]

One more from PFA, a GPU for AI.  https://devblogs.nvidia.com/parallelforall/new-pascal-gpus-accelerate-inference-in-the-data-center/

[Dr. MD MD]

This thread reminds me of being back in college hanging out with my hacker buddies in the computer department. Whenever we all hung out the conversation would eventually get around to computer spec talk. A more philosophical friend of mine mused during one of those conversations that in our father's generation they all sat around and talked about how fast their cars were. In ours we talk about how fast our computers are. In a generation we went from the speed of physical travel to the speed of mental travel being the most important thing. Well, his words have never really stop echoing in my mind because they've only seemed to get more true as time goes by.

[(Sandman) Logan-5]

Hi Doc.   I'll bet a generation or two before that it was the fastest horse, best breeding, stamina, and rough trail traversal.   But I get your point.

W_A_N, the systems you're familiar with get an honorable mention in the following article. https://www.quora.com/Are-there-any-motherboards-designed-for-high-performance-computing


MFM, I found a good one for you. I'd still recommend server / enterprise level hardware if you are doing a production build rather than a hobby build  - especially the motherboard.      http://www.logicalincrements.com/articles/videoediting/ 
             &
https://turbofuture.com/computers/Build-Photo-Editing-PC

Check the comments on these pages too. There are additional tricks listed.

For example, I like this guy, and he is correct if you use the XEONs he mentions. ( If you are considering this, you're not the first as X58 Chipset components seem to be drying up.)

" Ryan 19 months ago

Consider going back in time to the x56.. Processors with the lga1366/socket Asus server mother board. 1. You can practically blow any I7 out of the water with two x5680s for $300 and you'll have more pcie slots to play with. Having DDR4 would be nice, but it's not a practical option, and probably won't be, for a while for budget workstations.  "             

    ^^^^        To avoid confusion, the lga 1366 motherboard socket uses an X58 chipset. His mention of x56.. refers to the intel XEON CPU designation e.g.  x5670, x5680, x5690, etc..  x=power draw (L would be low power) the 56 is a hexa-core (12 w/ hyperthreading) if it were 55 it would be a quad-core (8 w/ HT) The 55's have an 8MB cache whereas the 56 has 12. (Level-3)
Another tip on XEONs, stick with the ones that end in a 0. The specs are different with the interim numbered chips. Ex. Choice between x5674 and an x5670. pick x5670 unless they are the same price and check to see if the additional power draw is worth the minimal computational gain. most times it's not. Those are special purpose chips and you need to check the specs.

Here's a handy-dandy CPU comparison chart w/ hyperlinks to take you to the specs.  https://cpubenchmark.net/cpu_lookup.php?cpu=Intel+Xeon+X5670+%40+2.93GHz&id=1307

[Hog]

Last high end system I worked on when I was in Server development:

x3950 X6

OK its only 50 large, can I stream Gabcast on it, or doesnt it come with speakers?

peace
Hog

[WOTR]

This thread reminds me of being back in college hanging out with my hacker buddies in the computer department. Whenever we all hung out the conversation would eventually get around to computer spec talk. A more philosophical friend of mine mused during one of those conversations that in our father's generation they all sat around and talked about how fast their cars were. In ours we talk about how fast our computers are. In a generation we went from the speed of physical travel to the speed of mental travel being the most important thing. Well, his words have never really stop echoing in my mind because they've only seemed to get more true as time goes by.

I really like the thought.  I would only change the term "mental travel" to the "speed of information".  Note I did not say knowledge (though it can be used for such.)  Right now, the majority of processing power and internet speed for individuals is spent on netflix and porn.

That criticism does not detract from the argument your friend made in any way.  It is an interesting point to ponder...

[Dr. MD MD]

I really like the thought.  I would only change the term "mental travel" to the "speed of information".  Note I did not say knowledge (though it can be used for such.)  Right now, the majority of processing power and internet speed for individuals is spent on netflix and porn.

That criticism does not detract from the argument your friend made in any way.  It is an interesting point to ponder...

True but movies and fapping are important too. I was speaking more figuratively, like Steve Jobs' bicycle for the mind. 

https://www.youtube.com/watch?v=ob_GX50Za6c

[starrmtn001]

Hi Doc.   I'll bet a generation or two before that it was the fastest horse, best breeding, stamina, and rough trail traversal.   But I get your point.

W_A_N, the systems you're familiar with get an honorable mention in the following article. https://www.quora.com/Are-there-any-motherboards-designed-for-high-performance-computing


MFM, I found a good one for you. I'd still recommend server / enterprise level hardware if you are doing a production build rather than a hobby build  - especially the motherboard.      http://www.logicalincrements.com/articles/videoediting/ 
             &
https://turbofuture.com/computers/Build-Photo-Editing-PC

Check the comments on these pages too. There are additional tricks listed.

For example, I like this guy, and he is correct if you use the XEONs he mentions. ( If you are considering this, you're not the first as X58 Chipset components seem to be drying up.)

" Ryan 19 months ago

Consider going back in time to the x56.. Processors with the lga1366/socket Asus server mother board. 1. You can practically blow any I7 out of the water with two x5680s for $300 and you'll have more pcie slots to play with. Having DDR4 would be nice, but it's not a practical option, and probably won't be, for a while for budget workstations.  "             

    ^^^^        To avoid confusion, the lga 1366 motherboard socket uses an X58 chipset. His mention of x56.. refers to the intel XEON CPU designation e.g.  x5670, x5680, x5690, etc..  x=power draw (L would be low power) the 56 is a hexa-core (12 w/ hyperthreading) if it were 55 it would be a quad-core (8 w/ HT) The 55's have an 8MB cache whereas the 56 has 12. (Level-3)
Another tip on XEONs, stick with the ones that end in a 0. The specs are different with the interim numbered chips. Ex. Choice between x5674 and an x5670. pick x5670 unless they are the same price and check to see if the additional power draw is worth the minimal computational gain. most times it's not. Those are special purpose chips and you need to check the specs.

Here's a handy-dandy CPU comparison chart w/ hyperlinks to take you to the specs.  https://cpubenchmark.net/cpu_lookup.php?cpu=Intel+Xeon+X5670+%40+2.93GHz&id=1307

Why do I even come here?

You're good man, Logan, but every time I read your stuff in your thread, I fry a number of neural circuits.

I really want to learn this stuff, but I just don't know where to start.

[Walks_At_Night]

OK its only 50 large, can I stream Gabcast on it, or doesnt it come with speakers?

peace
Hog

No speakers and 50 G's just gets you in the door.  You'll want it fully populated to bring it
up to it's 192 CPU core and 12TB of RAM potential.

So you'll need some extra goodies:
extra CPU books and heat sinks. eight Intel Xeon E7-8800 CPU's, 192 64GB DIMMS,
32 SSD or SAS drives with a couple of storage HBAs to drive them,  optics for the onboard 10Gb ethernet
ports, extra I/O books for PCIe adapters  and of course extra power supplies for redundancy.

I've took a quick run thru the online configurator and got you a new quote.  Pretty insane.   

 

[Walks_At_Night]

Hi Doc.   I'll bet a generation or two before that it was the fastest horse, best breeding, stamina, and rough trail traversal.   But I get your point.

W_A_N, the systems you're familiar with get an honorable mention in the following article. https://www.quora.com/Are-there-any-motherboards-designed-for-high-performance-computing


MFM, I found a good one for you. I'd still recommend server / enterprise level hardware if you are doing a production build rather than a hobby build  - especially the motherboard.      http://www.logicalincrements.com/articles/videoediting/ 
             &
https://turbofuture.com/computers/Build-Photo-Editing-PC

Check the comments on these pages too. There are additional tricks listed.

For example, I like this guy, and he is correct if you use the XEONs he mentions. ( If you are considering this, you're not the first as X58 Chipset components seem to be drying up.)

" Ryan 19 months ago

Consider going back in time to the x56.. Processors with the lga1366/socket Asus server mother board. 1. You can practically blow any I7 out of the water with two x5680s for $300 and you'll have more pcie slots to play with. Having DDR4 would be nice, but it's not a practical option, and probably won't be, for a while for budget workstations.  "             

    ^^^^        To avoid confusion, the lga 1366 motherboard socket uses an X58 chipset. His mention of x56.. refers to the intel XEON CPU designation e.g.  x5670, x5680, x5690, etc..  x=power draw (L would be low power) the 56 is a hexa-core (12 w/ hyperthreading) if it were 55 it would be a quad-core (8 w/ HT) The 55's have an 8MB cache whereas the 56 has 12. (Level-3)
Another tip on XEONs, stick with the ones that end in a 0. The specs are different with the interim numbered chips. Ex. Choice between x5674 and an x5670. pick x5670 unless they are the same price and check to see if the additional power draw is worth the minimal computational gain. most times it's not. Those are special purpose chips and you need to check the specs.

Here's a handy-dandy CPU comparison chart w/ hyperlinks to take you to the specs.  https://cpubenchmark.net/cpu_lookup.php?cpu=Intel+Xeon+X5670+%40+2.93GHz&id=1307

Hey - the old x366 is on that page!  Cool - a *very* well built and designed machine

[(Sandman) Logan-5]

Why do I even come here?

You're good man, Logan, but every time I read your stuff in your thread, I fry a number of neural circuits.

I really want to learn this stuff, but I just don't know where to start.

There are some techies here, but for the rest who want to learn (or the techs that want to brush up,) I've collected and tested some links that work with no Javascript required, and no sign up needed.
These are free resources available on the web to get you started. (It will take some time to go through these and the interlinks they provide, but if you want to learn, take a weekend and some bandwidth and have at it.    )

http://www.openculture.com/free-computer-science-textbooks  - very good portal. Also check the inter-link for free         
                                                                                                                   online courses (first paragraph.)

http://www.freetechbooks.com/#collapse11

CS & Programming Meta-Link: https://github.com/vhf/free-programming-books/blob/master/free-programming-books.md
                           This is one of the best links you'll find. Covers almost everything

                            From the Meta-Link as pertaining to this thread...  http://book.mixu.net/distsys/single-page.html


There is absolutely no reason you should have to pay for an education. 

[(Sandman) Logan-5]

Folders - check this out - 2009. (especially Pgs 6-7)
http://www.bit-tech.net/bits/2009/08/03/how-to-build-the-best-folding-rig/1

Page 7 - all parts listed (except GeForce (upgrade to 750Ti) ) are still available. This unit can be built for just under a grand.

4x NVidia GeForce GTX 295 (not available) - Upgrade to 750Ti - $107 ea - $428 - Amazon
MSI K9A2 Platinum Motherboard - New Egg & Amazon N/A  eBay NIB   $ 150
AMD Phenom II X4 810 - $40 - Amazon
(2GB) x1 Corsair XMS2-8500 RAM - $26  - Amazon
250GB Hitachi HDT721025SLA380 - $40 - Amazon
1 KW Enermax N/A - Upgrade to Platimax 1350W 89+ efficiency PSU - $262 - Amazon

Total: $946.00

It can probably be built cheaper with a generation or two newer hardware as the parts listed in the article are obsolete and hard to find. The price bell curve for these parts is on the upswing.
 

[starrmtn001]

There are some techies here, but for the rest who want to learn (or the techs that want to brush up,) I've collected and tested some links that work with no Javascript required, and no sign up needed.
These are free resources available on the web to get you started. (It will take some time to go through these and the interlinks they provide, but if you want to learn, take a weekend and some bandwidth and have at it.    )

http://www.openculture.com/free-computer-science-textbooks  - very good portal. Also check the inter-link for free         
                                                                                                                   online courses (first paragraph.)

http://www.freetechbooks.com/#collapse11

CS & Programming Meta-Link: https://github.com/vhf/free-programming-books/blob/master/free-programming-books.md
                           This is one of the best links you'll find. Covers almost everything

                            From the Meta-Link as pertaining to this thread...  http://book.mixu.net/distsys/single-page.html


There is absolutely no reason you should have to pay for an education. 

Thank you very much, Logan.

[(Sandman) Logan-5]

Thank you very much, Logan.

Glad somebody can use them.

[theONE]

Thank you very much, Logan.

Talking about "High Performance Computing" in this thread -
I'm going to hack BellGab's resources to get to the pictures of starr's sexy legs pictures that she posted here few mont's ago.
Anyone has a password ?? --so I don't have to use brute force 

[starrmtn001]

Talking about "High Performance Computing" in this thread -
I'm going to hack BellGab's resources to get to the pictures of starr's sexy legs pictures that she posted here few mont's ago.
Anyone has a password ?? --so I don't have to use brute force 

Oh for God's sake.  Here!

[onan]

Oh for God's sake.  Here!

I need a smoke.

[starrmtn001]

I need a smoke.

If you're using kitchen matches, you can strike it on my legs!

[theONE]

Oh for God's sake.  Here!

If you're using kitchen matches, you can strike it on my legs!

hahaha, LMAO

[attachment deleted by admin]

[(Sandman) Logan-5]

HP is sporting a photonic super computer now that uses a unique form of memory transfer. Interesting.

https://fossbytes.com/hp-supercomputer-8000-times-faster/

http://www.extremetech.com/extreme/196003-hp-reveals-more-details-about-the-machine-linux-os-coming-2015-prototype-in-2016


Oh, and Big Blue is opening up Quantum Computing to the public. 

http://m.ibm.com/http/www-03.ibm.com/press/us/en/pressrelease/49661.wss

[starrmtn001]

If any of you "Geeks" have Netflix Steaming, ya gotta watch this.  Ya just GOTTA!  It is AWESOME!!!


Lo And Behold: Reveries Of The Connected World
https://www.netflix.com/watch/80097363?trackId=13641907&tctx=1%2C0%2Cc3e7ab1f-3788-4f4d-b95b-49d76143d1d5-118166871