My Big Design Up Front problem

Hi,
I would hardly call myself a veteran of Agile or Scrum, but I have been involved in a few projects that used variants of the methodology. I would classify myself as the product owner, if I had to pick a role.

Here is my situation:

We ran two major projects concurrently. Both were very similar. Technically, we should have built one underlying platform to provide a basis for both products, but we didn't and my regret for not having forced this is running very high.

We used elements of Scrum in both cases. Because each project was massive, we needed two vendors. Because I had a good perspective on the overall vision and had lived the hell of not having these systems, I had a reasonably well abstracted sense of the pillars of the platform we needed to develop. When I say "pillars", I mean the basic abstractions that, if acknowledged, could create a shared understanding and a shared vocabulary and (ultimately) a whole lot less code. Unfortunately, when I began to work with Vendor 1 they had no interest in hearing about my pillars. They wanted to work from a simple vision and hundreds of user stories that made no attempt to abstract any concepts that could add to comprehension, achieve efficiencies through reuse, etc.. They claimed that emergent design and refactoring would do the trick. Well.... it is two years later and we still don't have time to refactor and the complexity of the poorly abstracted code means we probably never will.

In the second project with another vendor, I created a 10 sentence document that provided a vision and, more importantly a set of abstractions and a vocabulary for the product. The document defined some abstractions/entities that I insisted we reference in our stories. So, when I talked about a "Clinical Model Score" for a user, or a "User Message", or a "Tailoring Pipeline" everyone knew what it meant. Even more important, we didn't labor with emergent design and, instead, just coded these pillars or constructs early on (in about 30 days) and then built on top of them. I improperly called this "iteration zero" at the time. Basically, we agreed to a framework.

The second project was far more successful than the first and I attribute it to having the pillars of the platform identified early. These were easy abstractions to make for a business person like myself, but they would have been nearly impossible for our teams to have come up with through their normal Scrum process. Refactoring is a great theory - and has worked for superficial aspects of our products, but not for critical abstractions that I am calling pillars or building blocks.

I certainly see the benefits of Scrum, and I'm not attempting to debate them. I also see issues with Big Design Up Front, but my design phase is about 2 weeks of thinking and 30 days of coding for a project of about 5 person years. My problem is that I am heading into another project with Scrum purists and I'm hoping to encourage the approach that has worked for me in the past. Here is what I am looking for:

-I know that there are a number of variants of Agile and Scrum.
-Is there a variant that accommodates the approach I took in the second project?
-If not, is there a good book or site that debates this in a balanced way? I have a basic understanding of Emergent Design, but I don't agree with it on its own and am looking for something that mixes this with Small Planning Up Front.

Sorry for the long post, but this is on the verge of becoming a multi-million dollar problem for my company.