CUDA, GPUs, and all that

Posted on June 20, 2018 by gcochran9

Looking for free sites that would allow some experimentation with GPU programming, likely CUDA.  Also, thinking about buying something, what would you recommend?

Something useful but not bleeding-edge?

This entry was posted in Uncategorized. Bookmark the permalink.

17 Responses to CUDA, GPUs, and all that

  1. Jerome says:
    June 20, 2018 at 5:15 pm

    Really depends what you want to do. You can get started with CUDA on the Nvidia card in a high-end laptop or desktop. Good place to learn the ropes. If you decide you want more number-crunching power, get a gaming PC, or put a high-end GPU in your desktop. I would definitely start at the lower level before you throw serious money at it, so you’ll know what you’re getting.

    This is what I do for a living, feel free to contact me. You’ve got my e-mail.

    Reply
      • Jerome says:
        June 20, 2018 at 6:44 pm

        I am assuming, perhaps wrongly, that GCochran9 has some programming experience, but zero GPU experience. So I’m just saying that he probably already has a perfectly acceptable platform for learning GPU programming, he doesn’t need anything high-end to get started. But the question of what to do first depends upon his programming background and his ultimate goals. Which he may not want to discuss on his blog.

        Reply
        • gcochran9 says:
          June 20, 2018 at 8:59 pm

          Lots and lots of programming experience, zero GPU.

          Reply
          • Jerome says:
            June 21, 2018 at 7:50 am

            If you have C/C++ experience, you should have no problem with CUDA. You probably have an adequate GPU on your PC already. Download Nvidia’s free CUDA Toolkit. It has loads of examples you can fool around with, some very simple and some quite complicated. I suggest Ubuntu linux, but Windows works too. Then you’ll have a better understanding of what the buzzwords mean, and what the options are for your projects. As others have noted, you can buy a lot of CUDA pretty cheap if you go used.

            CUDA is Nvidia-specific, and CUDA apps won’t run on other hardware. But if you are just looking to write code for your own use, that should be no problem.

            Reply
  2. blur says:
    June 20, 2018 at 5:16 pm

    If I’m reading this correctly, the free trial here should give you up to 1360 hours with a Tesla K80:
    https://cloud.google.com/gpu/

    Reply
  3. gwern says:
    June 20, 2018 at 5:44 pm

    Google Colab ( https://colab.research.google.com/notebooks/welcome.ipynb ) gives you free access to a GPU. Some of the deep learning online courses like fast.ai sometimes come with cloud credits for GPU instances.

    If you want to experiment with deep learning in particular, probably best to start off with some pre-emptible AWS or GCE instances for the first few months and only pull the trigger on a deep learning box when you’re sure. Tim Dettmers’s GPU guide is good but a bit outdated. You should probably go with a 1080 or 1080ti because otherwise the lack of VRAM will quickly become a constraint for larger models and hopefully you’re not so poor you are price-constrained…

    Reply
    • cambrianexplosion2 says:
      June 21, 2018 at 1:56 am

      Gtx 1070 and 1070ti have same 8gb vram as 1080 but are cheaper and not too much slower. If you are going to build or buy a pc if you get an sli capable motherboard and high enough power supply you can run two gpus to run two models at once while still staying in the “consumer” price range. Check buildapcsales reddit and slickdeals for good deals on this stuff posted daily.

      Reply
  4. masharpe says:
    June 20, 2018 at 7:29 pm

    Depending on what you’re aiming to do, CUDA may be more low-level than necessary. For example, for machine learning, it’s so much easier to use a library like PyTorch (or TensorFlow). These libraries are quite general as far as what computation they allow you to do. (It seems like you’ve used numpy before, and PyTorch is roughly-speaking numpy for GPUs.)

    Reply
  5. Space Ghost says:
    June 20, 2018 at 10:30 pm

    Are you doing deep learning or do you just need to multiply a whole bunch of matrices? On some level they’re the same thing, but depending on what you’re doing CUDA might not be the right choice. Something like Keras can be good if you just want to iterate quickly on models.

    Reply
  6. TerenceTrent says:
    June 21, 2018 at 8:18 am

    People whose visual cortex has been destroyed sometimes have ‘blindsight’ where they can still unconsciously ‘see’ things – for instance, they will flinch if an object swings in front of their face. This might be due the rest of the brain being able to perceive things – in a primitive way – without the need for the more advanced visual cortex. The GPU is the visual cortex, and the computer the rest of the brain. Primitive creatures probably don’t need this GPU. (Pre-domesticated human types like Australian Aborigines have larger visual cortexes I think)

    You could probably make a bit of money investing in processors which are analogous to other parts of the brain.

    Reply
  7. Luke says:
    June 21, 2018 at 10:45 am

    Per above, if you’re looking for new tool in your toolbox, go for it. If you’re looking for an effective way to scale up the datasets you can explore, there may be better options.

    See, all the Tensor Flow stuff which is a little proprietary, and the new AWS Sage. Which thing less so as it supports the de facto standard, Jupyter Notebooks.

    Reply
  8. sprfls says:
    June 21, 2018 at 11:35 am

    No point in spending money on your own hardware. Instead I highly recommend checking out Paperspace. It’s better and cheaper than AWS or GCP. They offer the fast.ai course that gwern mentioned built-in as a machine template. They also recently came out with a new product (gradient) that allows you to quickly and easily run jobs and generally mess around in your code while all the GPU compute happens in the background.

    https://www.paperspace.com/&R=70qgs3

    ^^ If you sign up with this code (70qgs3) we both get free credits. I will donate any credit I receive back to West Hunter (that goes for anyone that signs up here).

    Also, depending on what you’re planning on doing, I could try to arrange for more credits. Sometimes they give them out for research or non-profit stuff. Anyways feel free to email me.

    Reply
  9. Eli says:
    June 21, 2018 at 5:29 pm

    Graphics? What devilishness are you getting up to, Cochran?

    Reply
  10. albatross says:
    June 21, 2018 at 6:35 pm

    Videomancy to hide the counter-lurk?

    Reply
  11. Michel Rouzic says:
    June 24, 2018 at 2:41 am

    It depends on what you want to do, but if you want to get started any compatible computer will do. I use OpenCL (a bit less popular than CUDA but works with non-nVidia hardware and maybe simpler, and it works on my laptop’s Intel GPU too) and an entry level nVidia GPU (I have a $80 GTX 750 Ti, not very good for gaming but perfectly fine for my OpenCL work).

    The most important thing to know about GPGPU programming is that your approach is everything, many people try to port their normal code and find that it doesn’t work so fast. That’s because you have to rethink everything, not only due to the parallel way it works, but also how memory lookups are so slow and math is so fast that you’ll want to keep memory operations to a bare minimum and make sure your threads read the same things at the same time. Also taking into account overheads like transferring data between the CPU’s RAM and the GPU’s memory, or queuing kernels, running each kernel having some overhead so you don’t want to enqueue tens of thousands of kernel executions a second. For instance if you process images instead of running one operation on a full image after the other you should rather send a sequential list of operations to run on a pixel so that you do them all in one go and only have to write the result to memory once, this might well run a hundred times faster than with a naive approach.

    So before you invest in a 1080 Ti try with whatever you have if it’s compatible, an entry level GPU might well be enough. Think about what you’d say if someone asked about buying a CPU for signal processing programming, depends on what they need but it’s very likely they don’t need all that much.

    Reply
  12. Alex says:
    June 24, 2018 at 9:43 am

    It won’t be free, but you could try using Amazon web services. If you use spot instances, which you could easily do, I think it could be like 10-20 cents per hour.

    As others have said, if you want to primarily do linear algebra you might as well use something like cupy:

    https://docs-cupy.chainer.org/en/stable/tutorial/basic.html

    A tool like chainer or pytorch will have built in tools for computing gradients via automatic differentiation.

    Reply

Leave a reply to Eli Cancel reply