AI can now beat multi player Hold’em game (1 Viewer)

[Frogzilla]

More evidence of holdem is boring.

How does the AI do in Dramaha tho?

A game with more information via card removal, less decision points due to fixed bet sizing, and overall weaker opponent pool? I think AI would clean up with any half decent programmer.

 

[Frogzilla]

@ReallyGoodUsername @BGinGA @Nex

The heads up solutions have been around awhile via PioSolver are pretty insightful. Pluribus is able to craft strategies around multi-way flops, but any heads up situation has been nearly solved. I recommend the YouTube channel “Finding Equilibrium” for some hands broken down quite well.

Of note is that most situations Pio (and probably pluribus) employ a mixed strategy, where 75% checking 25% betting a hand in a given situation can be superior to always checking or always betting.

It’s also of note that if Pio (and probably Pluribus) measures that it has a range and position advantage it gets aggressive. If it is at a disadvantage it is more passive.

Lastly, it is balanced so that it has the mathematically optimal call, bluff, value raise frequencies, and board coverage for multiple runouts (hence leaving some unorthodox hands in some of the ranges at a low frequency).

Interesting stuff.

 

[RainmanTrail]

The difference between a program like Pio solver and a well designed reinforcement learning neural network like what Pluribus claims to be is that if they played against each other, they would be break even as the number of hands played approaches infinity, but if playing against random real world opponents, Pluribus would likely yield a considerably higher win rate.

Most winning players figure out how to plug their obvious leaks and win by capitalizing off the mistakes of their weaker opponents. But the best players take it a step further by creating opportunities for their opponents to make even bigger mistakes. Pio solver is incapable of the latter, albeit itself incapable of being exploited in such a manner.

 

[Changster]

 

[mipevi]

I'm currently working on two PLO bots!

 

[BGinGA]

 

[Changster]

 

[hdgeno]

Prolly be no contest in a kickboxing match...

 

[Lemonzest]

I've been working on writing a very similar program that can learn to play any variation of poker ever since I closed down Chip Donkeys. I write code like this for work, and have been wanting to do this for years (but it's a lot of work and I have a one year old, lol). The most impressive part to me is the minimal compute power they were able to do this on. I would love to peek behind the curtain and see how they coded some of their algorithms to be so efficient. The concepts behind this aren't new though. It's reinforcement learning /Q-learning with recurrent neural networks, which is the same ML architecture behind alpha zero which beat the best Go players in the world after playing against itself for only a few hours. It doesn't need to know strategy at all. All it needs are options and outcomes. I love that it donk bets too; a topic I've been arguing about with some of my pro friends for decades. Exciting times for an AI geek like me.

I know nothing about code but I am enough of a nerd to also want to know more about the inner workings. I would be interested to know how your own poker coding project goes. Where is a good place to start for a total novice in terms of coding. There really is no alternative...coding and computing is the future.

 

[Lemonzest]

Uh, fake news? Here’s Pluribus’s graph:
View attachment 311047
Pluribus LOST 7 BB/100 over the 10,000 hand sample. Feels like a relevant fact. Maybe more relevant than “won $1,000 per hour”.

The research team “adjusted for luck”..which is way more in depth than the fairly standard assignment of all-in EV for those situations, and what do you know, their bot “won $1,000 an hour”. Is their luck adjustment valid? Possibly, but I get skeptical when the same team that is touting victory is the one that created the convoluted measurement methods. And any article that says Pluribus won without an asterisk is bullshit.

Great point. To say anything definitive the sample size would need to be massive.

 

[Nex]

I know nothing about code but I am enough of a nerd to also want to know more about the inner workings. I would be interested to know how your own poker coding project goes. Where is a good place to start for a total novice in terms of coding. There really is no alternative...coding and computing is the future.

http://shop.oreilly.com/product/0636920028192.do

This is probably the best book for absolute beginners I've stumbled across so far that teaches the basic mindset you need as a software developer, and some first practical coding skills in an easy to use scripting language.

However, it's a loooooooong way from what you'll know after working through this book, to what you need to know to build halfway sophisticated "AI" using machine learning. Plus, the most popular tools for machine learning software development have been made for a different programming language. Learning a new language is not a too big deal for a professional software developer, they already know what they want to express and merely need to re-learn how to express it in that language... but for a novice that hasn't even mastered one language and the core concepts yet it may be devastating.

 

[Lemonzest]

http://shop.oreilly.com/product/0636920028192.do

This is probably the best book for absolute beginners I've stumbled across so far that teaches the basic mindset you need as a software developer, and some first practical coding skills in an easy to use scripting language.

However, it's a loooooooong way from what you'll know after working through this book, to what you need to know to build halfway sophisticated "AI" using machine learning. Plus, the most popular tools for machine learning software development have been made for a different programming language. Learning a new language is not a too big deal for a professional software developer, they already know what they want to express and merely need to re-learn how to express it in that language... but for a novice that hasn't even mastered one language and the core concepts yet it may be devastating.

Thanks! I am interested in coding and software development in general. I don't
have lofty goals of writing AI script so the book you referenced is probably just
right !!

 

[Nex]

Thanks! I am interested in coding and software development in general. I don't
have lofty goals of writing AI script so the book you referenced is probably just
right !!

Even if you're not in to write highly complex machine learning AI some day but "only" good, robust standard software... you'll be in for years and years of extensive learning and experimenting. Also prepare to sacrifice some $$$ for specialist books later down the road. Particularly the standard references for software architcture are very expensive. (Luckily still peanuts compared to making custom CPCs though)

IT does not really need more people in general, but it direly needs more good people. Thing is, you never know if someone will be good until they've taken a dive into it. Some people can think like a machine. Many can't. Some have the foresight and discipline you need to produce solid work, many don't.

I hope I haven't scared you away from it now. It's fun and feels great being able to harness the incredibly big computational power of a modern machine to do precisely what you want it to do. Making it your job does take some of the fun away from it, but if you keep it a hobby...

 

[WedgeRock]

 

[Lemonzest]

Even if you're not in to write highly complex machine learning AI some day but "only" good, robust standard software... you'll be in for years and years of extensive learning and experimenting. Also prepare to sacrifice some $$$ for specialist books later down the road. Particularly the standard references for software architcture are very expensive. (Luckily still peanuts compared to making custom CPCs though)

IT does not really need more people in general, but it direly needs more good people. Thing is, you never know if someone will be good until they've taken a dive into it. Some people can think like a machine. Many can't. Some have the foresight and discipline you need to produce solid work, many don't.

I hope I haven't scared you away from it now. It's fun and feels great being able to harness the incredibly big computational power of a modern machine to do precisely what you want it to do. Making it your job does take some of the fun away from it, but if you keep it a hobby...

My exploits would purely be for my personal side projects. I already have a career which I am happy with. Then maybe if any of my projects takes off I would no longer need my day job - that sort of thing. Coding would just be a means to an end. I have zero interest in working for someone else where coding is my 8-5 grind. I have ideas for apps and I would prefer to do just do it all myself rather than hire out someone and get thousands of dollars into a project with a contractor.
I actually have so many ideas and I am getting bored of playing video games in the evenings.

I did some very basic coding back in high school and I don't even remember what language it was. I know what you mean about thinking like a machine. I remember running the program and it wouldn't work so I would have to scour the code and then add a comma or something in the right spot. After that the program would run perfectly. If I am having fun with it then a few grand in books doesn't bother me.

 

[Nex]

 

[youvebeennamed]

https://www.nytimes.com/2019/07/11/science/poker-robot-ai-artificial-intelligence.html

The computer program learned to play pretty much the same way we do, by playing lots of hands. It played trillions of hands against itself in 8 days!! It then would evaluate the best play after each hand. It originally bluffed too much but the algorithm was adjusted and now it crushes.

I wonder, what will be the last activity that humans will be able to "beat" computers in?

 

[hdgeno]

Prolly be no contest in a kickboxing match...

I wonder, what will be the last activity that humans will be able to "beat" computers in?

See above

 

[Highli99]

Humans have a spark of irrationality that will always separate us from computers. Spite and altruism are examples of this. This makes us unpredictable.