A debut goal from Tom Mehew marked a memorable night for the youngster, who edged Rovers ahead with a fine long-range strike inside the first-half.
The visitors were able to find an equaliser as Zak Jules headed home from a corner with 30 minutes on the clock, as the balance to the scores remained poised at the break.
Rovers managed to find the equalising goal to bring the game back level, as a low finish from Jonah Ayunga from the edge of the area saw the striker grab his first Rovers goal and take the encounter to a penalty shootout to decide the winner of the bonus point on offer.
Harries and Kelly both saw their penalties saved, as Walsall took the extra point, with a 4-2 win in the shootout.
Heading into the encounter, Rovers boss Ben Garner opted to make seven changes to the side that faced Ipswich Town on the weekend prior.
Cian Harries, Michael Kelly, Josh Hare, David Tutonda, Tom Mehew, Zain Walker and Jonah Ayunga all came into the starting eleven for the group stage fixture at the Mem.
Walsall, managed by former Rovers Manager, Darrell Clarke, included a familiar face in the shape of James Clarke in the starting XI, with the defender skippering the Saddlers for the tie.
The early stages played out with little in the way of goalmouth action. A long-range strike from Jonah Ayunga was the closest both sides came to creating an early opening, as the effort on the turn from the Rovers forward climbed high over the crossbar.
There were was a scare for Rovers as the first half progressed. The opening games down the left, as cross fell into the path of Emmanuel Osadebe. The midfielder launched an acrobatic volley that took a deflection onto the upright, with Caolan Lavery snatching at the rebound and directing his effort high over the target from just yards out.
The chance for the visitors seemed to spark the tempo in the game, as moments later, Rovers mounted a credible chance of their own. The chance was crafted by good positioning down the left from Tutonda, as the left-back managed to break free in space down the launch a fierce driven cross that fizzed across the face of the goal and needed just a slight touch to turn the ball home.
Rovers took to the lead shortly after and managed to do so in some style. A move through the middle of the pitch saw Hargreaves square a pass to Tom Mehew, with the youngster taking aim from 25-yards out to find the bottom corner and to make his debut that touch more memorable.
The lead for Gas was short-lived, as Walsall drew level on the half-hour mark. Defender Zak Jules was able to lose his marker from a corner to plant home a header to bring the tie back level.
There was a half-chance for Cam Hargreaves to restore Rovers’ lead, as a neat twist and turn from the youngster on the edge of the area opened up a shooting opportunity, but the tenacious midfielder couldn’t keep his effort on target.
The rest of the first-half passed with both sides unable to force an opportunity to take a slim advantage into half-time, as the scoreline remained equal at the interval.
Following the break, the encounter took a lengthy spell to get back up to pace.
A dart forward from Cam Hargreaves on the hour mark led to the first opportunity of the final 45 minutes, as the youngster broke towards the edge of the Walsall area and directed an effort that continued to rise over the frame of the goal, where perhaps better options were to be had either side of him.
A flurry of substitutions from Walsall led to an instant breakthrough for Darrell Clarke’s men, as a Josh Gordon climbed to meet a cross from the right and was able to steer home a header into the far corner.
The goal from the visitors sparked movement from the Rovers bench, as Jayden Mitchell-Lawson and James Daly were both introduced.
With twenty minutes left on the clock, Rovers did manage to find a goal to bring the scores level. It was a direct run forward from Mitchell-Lawson that opened up the Walsall defence, as the substitute was able to work the ball to the unmarked Ayunga on the fridge of the area. The striker took advantage of a clear path to goal and rifled a low finish that darted into the bottom corner.
Ayunga’s goal took the tie to penalites, as neither side was able to score late to prevent a penalty shootout to decide who would claim the bonus point, as the 90 minutes ended tied at 1-1.
In the penalty shoot-out, early misses from Kelly and Harries meant the Gas lost out 4-2, missing out on the bonus point.
Rovers begin their Sky Bet League One fixtures with a trip to Sunderland on Saturday.
Rovers: Jordi van Stappershoef (GK), Alfie Kilgour (C), Cian Harries, Michael Kelly, Josh Hare, David Tutonda, Tom Mehew (83′ Josh Grant), Jack Baldwin, Cam Hargreaves (70′ James Daly), Zain Walker (70′ Jayden Mitchell-Lawson), Jonah Ayunga.
Subs: Alexis Andre-Jr (GK), Mark Little, Pablo Martinez, Sam Heal, Josh Grant.
Walsall: Jack Rose (GK), Zak Jules, James Clarke (C), Liam Kinsella, Caolan Lavery, Josh Gordon, Alfie Bates, Jack Nolan (57′ Rory Holden), George Nurse, Emmanuel Osadebe (57′ Wes McDonald), Hayden White (57′ Cameron Norman).
Subs: Liam Roberts (GK), Dan Scarr, Elijah Adebayo, Sam Perry.