Time: 2025-07-11 11:23:46 Source: Codora.ai Author: news Reading: 239 times

building blocks for your own AIworkloads a microservice will beconfigured to work in various ways withvarious Technologies and ai for music productionso here is abunch of um uh microservices they haveso for example on the left hand side ofhighlighted llms and so this issomething that you put in front of theTGI Service uh or VM or whatever isserving the model um but here You' goand you'd say okay I'm doing textgeneration so I want to go into thatfolder and I want to serve it via TGI umand then when you go into there you justsee a Docker container but the idea isthat you can see that they have it forall these different variant and all ofthese um microservices are like this andfor the examples it's very similar wherethey have these steps but it's brokenbased down on where you want to deployit and what Hardware you want to use sohopefully you see the value in that butwe will go to a Hands-On lab so we knowfor certain[Music]okay hey everyone this is ang Brown inthis video I want to take a look atutilizing Opa so um I'm logged to my adaccount and I figured what we can do iswe can go ahead and deploy something onan ec2 instance now we could do thislocally but um uh I don't know I justthink that generally if you're going tobe using Opa you're going to be puttinginto production and so I want to put itsomewhere where we can uh go ahead anddeploy it we probably also do this ingit pod but again we'll just go aheadand do that here so um in the GenEssentials course I don't have the codeyet but I'm going to go ahead and justcopy this link because I have built thisfor another workshop and we can bring itover and just execute it over here umand so this one is called I thinkgen the Gen Training Day workshops andin here in my intermediateWorkshop I have one specifically for Opaand what we're looking for in here isthe OPA shfile as this will get us uh mostly setup I also have in here a cloud formationtemplate which will launch it as wellbutum you know I think what we can do is wecan just go ahead and uh set this upmanually so I'm going to go ahead andjust grab this here and I'm going to goover to here and we'll make a newdirectoryhere calledOpa and I'm going to make a new folderhere calledbasic and I'm going to go ahead andjust uh create this Opa Dosh file hereokay and what we'll do is we'll walkthrough this step by step and get thisall set up okay um so what I'm going towant for this is I think like a newbuntu instance so I'm going to go overto well I'm just think about this for asecond because in this we can run whatany kind of all we want and I want torun this with llama 3.21 bilon instructand for that we need about four uh fouror um I think four gigabytes of RAM andwe know that because when we weretalking to Rola early in the course shewas talking about or later in the courseMaybe she was saying that um you know 1billion parameters come out to four uhfour um gigabytes depending on if you'reusing 32 uh float Precision which iswhat this model could have and so fourfour gigabytes of RAM is most likelywhat we'll uh need to utilize um I wantto deploy this to a Zeon processor Ithink um so I'm going to go take a lookhere and say ec2 Zeon processors and thereason I say that is that if we go lookupOpa right and we go into GitHubhere and we make our way over to therepositories and we go into the Genexamples and from here we go into uhchat Q&A which is what we're going todeploy here today uh it's going to havedifferent configurations of how we candeploy this and so we're going to doDocker compos here today but notice it'ssaying like where do you want to deployit you want to deploy it on gpus um ontoAMD uh Rock M or Intel I'm going to gowith Intel here and then we have CPUsand Gotti I'm going to go to CP pus wehave aipc which by the way I have wecould deploy it to that and we'll go toZeon so I generally want to deploy it toa um Zeon processor now uh Zeonprocessors are Enterprise levelprocessors but if you ever wondered whatwould cost to buy one yourself if I wasto look up aipc umDell because I actually just bought anew computer or my company just boughtme a new computer and I was trying toconvince uh boo to let me get a a Dell auh a dell aipc and he just said no youdon't need one um but I just thoughtit'd be interesting to show that becauseit actually comes with a Zeon processorI think it's like Dell AIPrecision see if I can find it here andso if we go over to here let me see if Ican findit um and we go toworkloads let's godesktops Precision fixed workstationsand Dells are really good computers tobuy they are a bit expensive but if youbuy their extended warranties they'rereally really awesome and so we havedifferent ones here but I specificallywant to go to the[Music]aips go to all workstations here for asecond here we go and so if we scroll ondown if we go to the workstations for AIso you know if you ever wondered if likecould you buy yourself a a Zeonprocessor you absolutely could if we godown over to here here and we have likethese Tower work stations there's allsorts of kind but if we went to thetower like 80 uh 7875 you notice it says4,784 it's probably mostly because ofthe GPU but if we go into one and we saycustomize andbuy um this one's using AMD so it's notnecessarily what I would like so I'mgoing to go back here as we want onethat specifically has um Intel and somaybe it isthe it might

(Editor in charge: python)

Previous article: Learn Blockchain, Solidity, and Full Stack Web3 Development with JavaScript – 32-Hour Course - Ep268
Next article: GenAI Essentials – Full Course for Beginners - Ep146

• 
• 
• 
• 
• 
• 
• 
•