I have never successfully tickled a trout. I learned as a lad that it takes too much time, and patience is certainly not one of my virtues. And after all, no matter how much effort you put in – whether it be making sure your shadow is not on the water or moving at a speed that does not spook the speckled fish – sometimes it really does all boil down to luck.

Trying to get into North Korea to get a story is rather similar. But the rewards for patience and perseverance, I now believe, far outweigh the achievement of flipping a trout onto the bank of a river.

Like many other journalists, I first visited Pyongyang at the invitation of the North Korean authorities in the spring of 2002. The government was on a charm offensive shortly before its two most entrenched historical and ideological rivals – South Korea and Japan – jointly staged the finals of the 2002 soccer World Cup.

North Korea's world-famous mass games were moved forward from their usual dates in the autumn, and the city was reportedly spruced up before any journalist who had ever expressed an interest in visiting the country was invited to stop by.

The entire trip was, unsurprisingly, choreographed to the last footfall. We were bussed to mass displays of adoration for the nation's leaders in the huge May Day Stadium, witnessed colossal armed march-pasts through Kim Il Sung Square and were treated to cultural events in which I failed miserably at the local equivalent of Morris dancing. I will, however, be able to tell my grandchildren that I saw Kim Jong Il salute his troops from the balcony of the Grand People's Study House.

At the airport, waiting for my Air Koryo flight back to China and thence to Japan, I sat opposite the head of the delegation that had received me. To my right sat the female translator who had patiently answered all my questions for the previous five days, and mentioned in conversation that she hoped her two teenage daughters would be able to go to good schools, serve the party and get good jobs when the time came.

My official host had never let on that he spoke English, so it caught me off guard when he leaned over the table and told me quite clearly that if I wrote anything negative about North Korea, then it would be "very bad for her." To make it quite clear what he meant, he pointed at my translator. Her face, to put it mildly, fell.

I hadn't exactly misbehaved on my trip, but I had briefly managed to shake the guard assigned to me as a "guide" for the duration – so I guessed the translator was seeing her daughters' lives flash before her eyes. And that the vision wasn't pretty.

Safely back in Japan, I wrote as many articles as I could wring out of the trip – geopolitical, analytical, observatory, even a travel piece – and tried not to think about the translator and her family.

So you do the trip, assume it's absolutely a one-off chance and start looking for the next big story. But it's not long before you accept that of all the big stories in this part of the world, North Korea is right up there. Whether it’s gulags, mass starvation, missile launches, nuclear-weapons tests or Kim Jong Il's Japanese chef, your boss wants to know when you might be able to get back there again. And you humor him and say you're looking into it – but assume that, as not everything you wrote last time might have delighted their powers-that-be, there is more chance of a candid one-on-one chat with the Japanese Emperor.

So we return to that immeasurable quality of luck.

Five years ago, I met a left-wing Japanese academic (who must remain anonymous), but who had been able to visit North Korea on four occasions to carry out research into a project that will culminate next year in an exhaustively researched book. The book is inevitably pro-North Korea – clearly he would never have been able to get access had it been otherwise – but it is equally a legitimate study that I believe provides utterly damning evidence that the people of North Korea have been the victims of atrocities.

Which brings us to a problem. The North Korean government has told so many lies, done so many terrible things – to its own people as well as abducting Japanese nationals, bombing South Korean airliners and seizing the odd South Korean movie director on the whim of the Dear Leader – as well as giving two-fingered salutes to the international community on so many occasions, that the world beyond its borders does not believe what it says.

It does not help that every single time its people utter the word "America," it is preceded by "imperialist." Ditto "Japanese," although that is often spiced up with "warmonger" as well. And while those descriptions may well be true, this is propaganda that is stuck in East Germany circa 1960. The regime's use of these terms turns them into caricatures. They really need a good PR man.

The professor's tales of his studies in North Korea got me thinking that he might be the way for me to get back into North Korea, accompanying him on his next bout of research. He agreed to put it to his contacts in Pyongyang. And there it languished. I made halfhearted efforts to resurrect the project, but assumed that I had been blacklisted after my first visit.

Another chance encounter, this time with an independent British documentary director, set the wheels in motion once again, helped immeasurably by the fact that Tim Tate was a regular contributor of programs to the Al Jazeera television channel.

And while trying to get into the country in the name of the ideologically unsound Daily Telegraph had failed, North Korea was apparently willing to let me in if I was with the more palatable Al Jazeera team. Or perhaps it was the fact that they would be making a full documentary on the story; Kim Jong Il is reportedly something of a movie and TV fan.

It still took a couple of years to get all the pieces in place, numerous meetings with representatives of North Korea who live in Japan and lots and lots of paperwork, but confirmation that the trip was finally on came in September. But even with that promise, there was still a little voice in the back of my head that told me that it would be scrubbed at the last moment or that the interviews and access we had been promised would not materialize. Without the chance to speak to the people we had identified as central to the story, we would effectively be tourists.

With the professor and his assistant, I met the Al Jazeera crew – director Tate and cameraman Rick Manzanero – in Beijing on Oct. 23, and we flew on to Pyongyang.

It was quickly apparent that this trip was going to be very different from the previous one. The professor was welcomed like an old friend after we came through the immigration gates, we were helped with our luggage and the video equipment, and there was not a heavy in sight among the four-member group that met us.

And so it continued for the five days we were fortunate enough to stay in North Korea. We were driven to sites around the country that would have been impossible to visit otherwise, we met ordinary people who told us extraordinary tales of what they had seen and experienced – and not once were they “coached” by the officials or did they seem to be reciting the party line for this particular story.

True, the museums reflect North Korea's unique take on history and the evidence is delivered in a clumsy and cliché-laden way, but there is a kernel of truth beneath all the bluster and bombast. Our guides laid on everything that we asked for – bar the exclusive interview with Kim Jong Il, unfortunately – and, remarkably, largely left us to our own devices.

I was able to walk out of the Koryo Hotel and stretch my legs for a couple of blocks. I attracted plenty of stares, but I never had a minder chase me down the street and usher me back inside the hotel, as had happened the last time I was there.

Pyongyang has changed little in the last seven years. There may be some newer cars on the roads – and Toyotas and Mitsubishis have been largely replaced by Mercedes sedans for the elite and Volvos or Fiats for second-tier VIPS – but the bicycle remains king in the city, and it’s ox-pulled carts and shoe leather in the countryside.

The people still wear Mao-style baggy suits in a selection of four colors, everyone has a badge in their lapel of the smiling face of the Dear Leader or the Great Leader, and roads outside of Pyongyang are more pothole than tarmac. The buildings are grandiose, bringing in the harvest is backbreaking work, and the only ones who smile at outsiders are young kids – and I saw one lad get a cuff round the ear from an adult for having the audacity to wave at us.

Is North Korea changing the way in which it handles foreign journalists? I doubt it. But I do believe that a good story that can be explained to the government as being not entirely negative to their way of looking at the world can happen. It just takes a lot of patience. ❶