How to plan long term if only a couple sprints are estimated, or how to choose between projects?

A basic principle of agile is to wait until the last responsible moment to flesh out and estimate stories. 

I checked the principles of the Agile Manifesto to see if I missed it, but that's a new one to me.

The Scrum Guide describes Product Backlog refinement as an ongoing activity to add details, such as a description, order, and size. It can be thought of as a team's look-ahead radar. Switching your radar on at the last moment may not be responsible -- it's best to avoid nasty surprises.

I'd suggest that refinement ought to be done sufficiently far ahead to allow the Product Owner to manage value effectively and for Sprint Planning to be de-risked.

My mistake! LRM is a principle of Lean, not agile in general. Nonetheless, the approach is valid - stories further out are less less fleshed out because they could change based on feedback from prior sprints. Must we estimate the whole backlog in order to forecast the end of the project?

What about in cases of projects to be chosen between? 

There is no agile principle of waiting "until the last responsible moment to flesh out and estimate stories". This does sound like one of the principles from The Toyota Way, which is to make decisions slowly by consensus-building and then implement the final decision quickly, which is referred to as nemawashi. There are some different flavors of exactly what this looks like like Lean Manufacturing, Lean Product Development, and Lean Software Development, but it's common to see something like this in lean implementations.

Generally speaking, it's good to defer refinement and estimation as long as possible. It can be wasteful if you spend effort on refinement only to have the work be deferred due to higher priority work. In a Scrum context, limiting the amount of the Product Backlog that is well-refined tends to be a good practice. How much of the Product Backlog should be refined is context-sensitive and depends on how much ambiguity, uncertainty, and change exists in the environment. A more stable environment can support refining a larger amount of Product Backlog Items than more uncertain and unstable environments.

Since a traditional burn-up chart does show the total work in-scope, it would require the body of work of interest to be reasonably well-refined and estimated. If you're estimating a couple of Sprints ahead and are still unable to get a body of work of interest refined and estimated, I'd want to understand why. Some of the problems that I've seen include too long between releases, bodies of work that are too large and should be split into smaller bodies of work, and a lack of focus on a singular effort.

You don't need all of the work estimated in order to choose between projects or bodies of work. Thinking that you can, before the work begins, have all of the stories estimated, leads to the trap of plan-driven development methodologies. There are contexts in which you cannot build reliable plans up-front due to the uncertainty, ambiguity, or emergence of new or modified requirements. You can estimate the body of work based on what is known, and perhaps even try to identify the known unknowns that may impact the work. Depending on what the work is, though, there may be unknown unknowns that will be discovered only by doing the work. The decision maker has to be empowered to make decisions with high amounts of uncertainty.

Must we estimate the whole backlog in order to forecast the end of the project?

I suggest that you read these books by Daniel S Vaccanti.  https://actionableagile.com/resources/publications/  The concepts have been very useful to me. The books explain how to use metrics to help a team become more predictable in their work and then how to use that predictability to forecast without having to estimate everything. The metrics used are based upon trailing indicators of work actually done rather than guesses made based upon the information known at the time of the guess.

I am happy to see you use the word "forecast" instead of "plan".  I prefer the forecast approach. And the method I use to explain my preference to others is by using the most common forecast known....weather forecast. 

A 3 month or 6 month weather forecast can give me a general idea of whether I should have an outdoor event.  But I am not going to be completely trustful or confident until the date gets closer.  The weather forecast for next week is going to be much more reliable because there is more relevant information known to use in making that forecast.  

 how can a decision maker reasonably choose between one project or another if she doesn't have all the stories for each estimated?

How about by using all of the information that they have. The story estimates are only a small part of an equation on whether to pursue a course of work.  How about the potential impact on your company's revenue?  How about the impact on the users' ability to be productive with your product?  How about the economic direction that can impact the need for the work? What about the potential that the work as you understand it today might change in the near future? What about the potential that the economy will change?  

The beauty of agile software development is that you start working on a problem as it is known today and deliver what the stakeholders need when they need it.  Often the what is known today is vastly different from what is actually delivered.   By doing, inspecting, and adapting you have the potential to change the timeline in both directions but you are delivering to the stakeholders what they want when they need it.  That is because they are involved and can tell you when to stop. 

Hi, Matthew!

In fact, as you say, we don’t choose between projects in agile through the estimation of the whole project or product. Why? Because we work with such complex products that trying to do something like that is simply not a possibility. 

Traditional Management believes just the contrary. We, on the other hand, think that working in that way is not different from what a fortune teller does.

So how do we choose between projects through an agile mindset? In a quite simple way, running the smallest experiment that would bring us enough information to clarify which option is best:

1. Decide what is the most important reason to choose one option over the other one.
2. Design the simplest possible experiment that will bring transparency about the best possible decision.
3. Decide which measures will be paramount in order to make a good decision.
4. Run the experiment.