Akhalkalaki is one of the bigger cities in southern Georgia. It is something of a travel hub, with connections to Tbilisi and Akhaltsikhe. It is ideal to visit Vardzia from here. The easiest way is probably to take a marshutka minibus to Khertvisi and try to catch one there to Vardzia. Taxis are available from Akhalkalaki.

From Akhalkalaki you can cross into Armenia to Gyumri. There are minibuses and taxis. A taxi to Gyumri should cost about 20 dollars.