You WERE almost jacked! A very similar incident happened to me at a gun range, several years ago. Same, 'song and dance' routine; same indecisive ambiguous behavior. In my opinion your caution and reserved aggression were warranted. This could, very easily, have turned out bad for you.
In my case I had, just, shot off the cylinder and two belt reloads with my Model 29, when - out of nowhere - this swarthy, well-dressed guy suddenly appears behind me; when I first saw him, he was standing next to my range bag! (He must have been watching me and deliberately staying out of my field of view.) He gave me a weird grin and remarked; 'You shot her dry; didn't you!' That was all I needed to hear; from that moment on, I had no doubt what was taking place. I immediately held the 44 magnum up with my left hand and showed him the open empty cylinder. As he looked at the Model 29 and smiled, I drew a pistol from my right hip, and said; 'Yes it's empty, all right; but this one is ready to go!'
Now this guy had, 'brass'. I could tell he was, 'phased' but not about to give up. (He, actually, made me feel like a, 'pigeon'.) He engaged me in a, sort of, jovial banter while steadily moving toward my gun hand. We, almost, completed a full circle together while he made small talk: He'd move to my right side; I'd shift my gun hand away from him; and he'd continue to move with me. He acted comical; but I could see the frustration in his eyes. I finally put one of the benchrests between us, cocked the hammer, and lay my hand down on the table with the muzzle pointing straight at him. I asked him; 'What kind of game are we playing, here?' His reply startled me. 'All I'm doing, today, is trying to pick up a birthday present for a friend.' Then he walked away from me; and I immediately threw my stuff in the truck and left the range. Shortly thereafter I bought my first cell phone; today, whenever I go out, I am never without it.
Now I'm the kind of guy who crosses every, 'T' and dots every, 'I'; but, in any situation like this, if you wait until that moment when you are absolutely certain, then, in my opinion, it will already be too late! I think you did the right thing. Oh, yeah, this guy had his left hand in his front pants pocket, almost, the entire time he talked to me. When he turned to leave I caught a glimpse of what I believe to have been a little, silver, Raven semi-automatic. I guess he didn't want to match it against the fifteen rounds held in my S&W Model 59. Smart choice!