• Welcome to BellGab.com Archive.
 

High Performance Computing

Started by (Sandman) Logan-5, March 19, 2016, 10:21:05 PM

This thread should be used for all aspects of HPC and Clustering. If you've built your own system and are proud of it, show it off here too - please include the extra goodies, specs, and bench tests if you did them.




The first question you need to ask yourself is: Are the programs you are running compute intensive, or memory / hard-disk intensive ? If you are running highly parallel programs like GIMP, POVRAY(not parallel coded), Scientific Computational Modelling, or numbers crunching programs, keep reading.

Some of the hardware factors that determine a computer's speed of execution are cpu frequency and # of cores, memory speed and rank, interconnect speed between components, disk write (and [not so much] read) speeds, PCI-e bandwidth,  NIC bandwidth and interconnect technology (copper, fiber, infiniband, etc.)

Traditional cluster computers dependended on multiple independant systems connected through an ethernet with switches controlling packet routing. Switches are expensive, contribute to latency, and are a wiring mess. 
What if you could get rid of the interconnect wiring and the switches, juice the ethernet and keep all the boards contained in a single tower with 4 nodes for under $6,000.00 ?

Worth checking out ?

One more thing, when you're not running highly parallel computational tasks, this system also doubles as a workstation. You could build yourself a system like this through lots of research, trial and error, and writing your own code to tie everything together, or you can check out the following links.

http://www.clustermonkey.net/Interconnects/experiments-with-switchless-10gige-the-bonded-loop.html - Lots of good clustering info.
Beowulf info + links - http://fscked.org/writings/clusters/cluster-1.html

-------------------------------------------------   The Limulus Project! â,,¢   -----------------------------------------------------

http://limulus.basement-supercomputing.com/

Buy one pre-built or get help designing one. 

Limulus Benchmarks

Base Model $5,495.00  here,  + Spec Sheet



-----------------------------------------------   Clustering Books   ---------------------------------------------------------------------------------

http://www.clustermonkey.net/Columns/Cluster-Newbie/


---------------------------------------------------------------------------------------------------------------------------------------------------------

Down & dirty use of old computers - Beowulf.

...And just so everyone that reads this thread has an understanding of how computers work - Don't let the magic smoke escape !

Explanation:

The core of a computer is a small trapped evil spirit (it is no coincidence that UNIX and GNU/Linux processes are called daemons!) This imp is imprisoned in your computer as punishment for something it did in the netherworld. Generally speaking, the more evil the imp is, the faster the computer is. The imps inside 1,000Mhz and slower computers are usually just big fat jerks - a.k.a. Sendas. There are also lesser ghosts and hobgoblins that inhabit the expansion cards and peripherals of your computer. These evil spirits are trapped within the hardware of your computer. Take a screwdriver, open up your computer, and take a look at the circuit boards. They are usually green, and are covered with complex patterns of thin copper lines. These are Circuit Runes, written in arcane and ancient languages that describe the magic spells that bind the imps to the chips. Be very careful with circuits. If you scratch off even the tiniest bit of the runes, the spell will be broken, and the imp will escape in a puff of magic smoke. If you coincidently happen to scratch these runes while the imp is awake and plugged to power, your fingers might burn and you would be thrown back with a shock. Well, that is the evil nature of the imps. The movie series Terminator is about what could happen if the evil of the imps in computers comes to life and take control. Also note the serial and part numbers printed on the boards. These identify exactly what sort of evil spirit it is, in case you ever have to order a replacement from the factory. :o




OK everyone. I may have overlooked something here. HPC runs Linux. If you want an M$ system, SQL Server has clustering built in, but you'll need to buy a yearly license. The last instance of M$ I've used was SQL_Server_2008_R2. HPC is mainly a Linux thing.

albrecht

Quote from: (Sandman) Logan-5 on March 20, 2016, 08:21:52 PM
OK everyone. I may have overlooked something here. HPC runs Linux. If you want an M$ system, SQL Server has clustering built in, but you'll need to buy a yearly license. The last instance of M$ I've used was SQL_Server_2008_R2. HPC is mainly a Linux thing.
I only downloaded your last post because I don't understand this stuff and, figured, a pdf would infect my XP computer.  ;) But seriously, being a person not about computers since BASIC and dBASE, I cleaned on old laptop and put on MINT (Linux, I guess based) for random, non-secure use (watch a movie etc and more of an experiment.) And old laptap Dell works very fast and can watch movies out on the deck etc easy with no problems. It didn't, somehow (maybe for better) get WiFi? But I don't really care.

zeebo

Quote from: albrecht on March 20, 2016, 08:31:41 PM
... since BASIC and dBASE ...

Nice.  I used to work in FoxPro, a cousin of dBASE I think.  High performance for it's time.   ;)

Quote from: albrecht on March 20, 2016, 08:31:41 PM
I only downloaded your last post because I don't understand this stuff and, figured, a pdf would infect my XP computer.  ;) But seriously, being a person not about computers since BASIC and dBASE, I cleaned on old laptop and put on MINT (Linux, I guess based) for random, non-secure use (watch a movie etc and more of an experiment.) And old laptap Dell works very fast and can watch movies out on the deck etc easy with no problems. It didn't, somehow (maybe for better) get WiFi? But I don't really care.
That's odd. Mint is usually pretty good about driver compatibility. :o
BTW, if all you're doing is watching movies, or typing up a document here and there, keep that puppy air-gapped (no working wi-fi). The alphabet-soup agancies and the 'Crackers' are known for remotely activating the built in camera and mic. (Anyone can using 'shodan' to get an IP and then using another system to explore & activate those IP addresses.)
You can always transfer any work you do on it to a flash drive and print or E-mail it on a connected system. ;)

OK, looks like I'm going to have to dig up some of the old Tut's on building your own HPC. It's actually kind of difficult finding good, up to date info on this subject. I'll see if I can re-locate them in the next couple of days. ;)

Quote from: zeebo on March 20, 2016, 09:44:40 PM
Nice.  I used to work in FoxPro, a cousin of dBASE I think.  High performance for it's time.   ;)
I've used FoxPro w/ VB6.0 to revamp a ticket selling app for Lambeau field. Yes, it was impressively quick for its' time.

The following information is dated, but it gives a decent idea of how to build a GPU compute cluster. Personally, I'd run PXE boot for the nodes (no HDD / SSD required on the nodes. )

http://www.computationalmathematics.org/topics/files/beowulf_gpu_cluster.html

Give me some time peeps, and I'll be able to get some better info. :)

If you are going to build yourself an HPC, alternative cooling methods might be something to consider. How about hooking up the equivelant of an air conditioner to your system ? It's called Phase-Change cooling and is very effective, but also expensive. The technology is actually developing and if you check the over-clocker sites, you can find cases w/ Phase change built right in.

The do-it-yourselfers out there might want to check this article out.

http://www.overclockers.com/build-your-own-phase-change-pc-cooling-system/

It's a hell-of-a-project.


 Gamers take note...
In 2010, The United States Air Force brought the "Condor" cluster Supercomputer online. It was built out of over 1700 PS3's and is very energy efficient. Cost: Approximately 2.5 million. (Cheap by DOD FWA dept. standards.)



http://www.govtech.com/technology/PlayStation-3-Providing-Supercomputing-to-Universities.html

http://www.popsci.com/technology/article/2010-12/air-forces-new-supercomputer-made-1760-playstation-3s

http://www.gamasutra.com/view/news/31784/US_Air_Force_Creates_Powerful_Supercomputer_Out_Of_PS3s.php

I've attached the AFRL performance report from 2011.



China's about to kick our ass again with their new accelerators. Direct Memory access on a Digital signal processor. This puppy would make one hell of a GPU.

http://www.nextplatform.com/2015/07/15/inside-chinas-next-generation-dsp-supercomputer-accelerator/

Forget Raspberri Pi, for just under double the price of a Pi, you can have this: http://www.adapteva.com/parallella-board/

This opens up a world of possibilities.

http://www.parallella.org/buy/


Thanks for this Sandman, I want to try and build something myself in the 5k range to make video animations or run an environment of virtual machines.


Quote from: Mind Flayer Monk on March 21, 2016, 03:17:18 AM
Thanks for this Sandman, I want to try and build something myself in the 5k range to make video animations or run an environment of virtual machines.
;)
I would suggest looking into flash accelerators. These are SSD's that run off the PCI-E bus and are 2-3x the speed of SATA III. The new ones are hitting ove 1 GB/second transfer speeds. Most are now also bootable - meaning you don't need an additional boot SATA SSD. There have been some problems running VM's from the early accelerators, but I think most of those problems have been worked out.


http://www.tweaktown.com/reviews/7062/intel-750-1-2tb-nvme-pcie-gen3-x4-aic-ssd-review/index.html Gen 3 is backwards compatible w/ Gen 2.


http://www.bit-tech.net/tag/pcie-ssd

You've got lots of research to do. Do you have the build planned out yet ?  The new cards are pretty kick-ass

http://www.tomshardware.com/t/graphics-cards/

You might also want to check prices on the Tesla M2xxx series.

There are some regular sellers on EBay for new, but for PCI-e SSD's, Amazon generally has better prices and warrantees. Make sure you're running a server board. I'd look at Supermicro for a MoBo, possibly 1 or two generations old (for best price / performance ratio.) Put your cash into the GPU's and RAM. There are still problems w/ the AMD APU. Go XEON.
The reason you want to consider a server MoBo is because they are designed for ECC RAM which will eliminate bit-flip errors if you are running multiple GPU's in SLI or Crossfire.
If you cruise the gaming and over-clocker sites, you'll find a lot of valuable information that also applies to workstations.
http://www.tomshardware.com/reviews/best-gpus,4380.html
How many monitors will you be using ?


PM me with the build specs if you'd like some help. ;)


In case you didn't catch it feom the specs, the M2090 has no output port for driving a display. If you wish to direct connect a Tesla, you'll have to stick w/ the 2070 series. ;)

Edit: ...or jump to the K series. (Current generation.) ;)

Here's an example of what can be found for under $100.00 on ebay.
10x MoBo's to be used as slave nodes. 2.2GHz, 2GB RAM, heatsinks.
All you need is power supplies.
Not fast, not new, no PCIe slots, not gauranteed to work, but for a cheap project to try out and see if you can set up a cluster (diskless slave nodes,) it can be done. Put the gpu's on the master node. Run these puppies as pure slaves.

http://www.ebay.com/itm/10x-HP-437794-001-Compaq-dc7800p-Ultra-Slim-Desktop-Motherboards-2-33GHz-DDR2-/131734072154?hash=item1eabf66b5a:g:xnkAAOSwB4NWyws5

listing good for another 22 hrs.



Quote from: Mind Flayer Monk on March 21, 2016, 03:17:18 AM
Thanks for this Sandman, I want to try and build something myself in the 5k range to make video animations or run an environment of virtual machines.
Take a look at this configurator to see what can be had with prebuilts.

https://www.pugetsystems.com/configure.php

Independant review of a puget system costing $4,500.00:  http://www.servethehome.com/puget-systems-genesis-workstation-experience/

Quote from: Mind Flayer Monk on March 21, 2016, 03:17:18 AM...
MFM - you may want to check out this System-On-Chip (SoC) Mobo for a head / Master Node for your build. ( CPU is built into the board. ) I read a different preview write up about these a while ago than the one I'll list, and they have tremendous potential with little power draw.

http://www.amazon.com/Supermicro-MBD-X10SDV-TLN4F-B-X10SDV-TLN4F-Mini-ITX-Motherboard/dp/B00Z8DBB0M/187-4549148-3504150?ie=UTF8&*Version*=1&*entries*=0

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182964

Review:  http://www.servethehome.com/supermicro-x10sdv-tln4f-review-platform/

One thing to note about this board, you'll need to buy a license to unlock the BIOS.  :(


If a big cluster is too much to tackle before you get your feet wet, check out this really nice tutorial on hooking up 4 Raspberry PI's to make a handy-dandy Distributed computer & DHCP Server all rolled into one.

For those of you that are asking yourselves "What the hell is a cluster, and why would I want one ?", check this out:

http://obscuredclarity.blogspot.com/2008/10/why-would-anyone-need-computer-cluster.html

area51drone

I know what clusters are and why some people need them... what I want to know is why you need one. 

Powered by SMFPacks Menu Editor Mod