Estimate deployment tasks, or not?

It's technical debt which must be remediated before an Increment becomes usable. Since the Product Backlog must tell the truth about all of the work that remains for the Product, I think that tells you whether or not it should be estimated.

My opinion is that If this is work that is done to improve the Product, it should be reflected in the Product Backlog. But from your description, I do not see this as work to improve the product.    

Every x sprints, there's a task to create the actual version we deploy to onpremise servers. This is a task we put in the sprint.

You are opting to treat it as a one-off task every 6 weeks.  I can't say that is the wrong way because only the people doing the work can form an opinion.  I will say that I have worked with teams that did similar and it worked effectively.  However, I do think it is wrong if the that entire Sprint is full of this type of work and does not contain anything that creates increments of work.  Each increment builds upon the previous and every Sprint should produce 1 or more usable increments.  Typically deployment tasks do not change the increment, they just provide a mechanism to move an increment into a space that is better accessible to the stakeholders. You shouldn't have a need to "create the actual version" to deploy because you would just deploy the latest usable increment. 

What your company does is create a quarterly release to production.  The Scrum framework does not say anything about the "how" of delivering to the stakeholders.  That is a procedural issue that is up to the organizations to decide. 

Ok, so we create releases every 3 months, but we want to release every sprint. 

The sprint we are making the release in, is not dedicated to such tasks, just an ordinary sprint, full of functional increments.

My view on the matter:

Creating a release, or any kind of mechanism to bring increments to users, in my opinion, and in our situation, is overhead that should be part of the DoD.

It's is not automated yet, but it should be. I believe we should either 2 things:

1 - give it storypoints and keep the task in all spints( the storypoints). estimate stays the aame

2 - We leave out the storypoints, and timebox it. That way, if we get more efficient by automating the release, less time is needed, and more time can be spent on stories. That way, de velocity goes up, when we automate the deployment.

What do you guys think?

If the team wants to get to automated deployments, why not include the work to do that in the Product Backlog and treat it as any other work that needs to be done in order to improve the product? Treat is just like a new feature and make the work visible.  Then it can be ordered appropriately within the Product Backlog by the Product Owner.  That would follow @Ian's recommendation.  

This is something that your Scrum Team needs to decide.  They are the ones for doing the work while making it transparent and visible.  Have you taken this to the team to discuss and decide.  Remember that no one can tell the Developers how they should work.  So let them be involved in this decision. 

Interesting question. Deployment seems like it can be perceived as not adding value; however, if you don't deploy your customer can't access the value, right? I like the concept of including it in the DoD but deployment is product or release level, not story level. I think creating a story for deployment is ok; interested to hear what others may say or alternatives proferred.

deployment is product or release level, not story level

Maybe for you but not for everyone.  If a single story provides value to the stakeholder and is something that needed to be done to the Product, why can't a single story be deployed?  I have in and alongside MANY teams that deploy stories on their own.  Remember that Scrum uses the concept of an Increment and one or more usable Increments should be completed in every Sprint.  Increments are actually what is deployed.  Increments also provide value.  So why would you not deploy value as soon as possible?  

Automating deployment processes absolutely provides value to a stakeholder.  That stakeholder is the Scrum Team.  By automating this it becomes easier for the Developers to deliver value.  That in turns leads to value that the stakeholders see easier and quicker.  Not all work in the Product Backlog has to be customer facing improvements.  The Scrum Guide states this in the opening paragraph for the section that defines the Product Backlog.

The Product Backlog is an emergent, ordered list of what is needed to improve the product. It is the single source of work undertaken by the Scrum Team.

Notice it says "what is needed to improve the product" but it doesn't say "what is needed to make the product better for the end user."   Stakeholders can be internal.  For example, if there is a need to upgrade certain libraries due to security concerns, your Corporate Security Team could be the stakeholder for those changes.