Usage Microsoft Team Server Foundation Scrum Template

As long as the Product Backlog is the only source of changes to the product and PBI's have their correct attributes, I don't see why this would be a bad thing.

From your description it seems that "Feature" is a group of backlog items, so the tool provides a way to group them to an upper level. A reason for that can be (from the scrum guide):

Multiple Scrum Teams often work together on the same product. One Product Backlog is used to describe the upcoming work on the product. A Product Backlog attribute that groups items may then be employed.

Also, 

I feel like it adds an unnecessary complexity.

I see why having two separate boards seems to be adding complexity, however, do the PO / DT feels the same way? As the SM this can be a subject of observation for future sprints and base your thoughts upon evidence before taking action (propose a change, raise the topic on a retro, etc).

Thanks Juan Prieto for your response.

The way I see it, Features in this sense is essentially what an Epic is, although I think we have a separate board for it as well.

Also, we're trying to follow the Nexus Guide since there are multiple teams working with our product but we have separate product backlogs for each team.

I want to align our process with what the Nexus Guide is saying but is this a deal breaker? We're not really sure if this will going to work since the organization has just adopted agile.

If the product is a single product, why is there more than one Product Backlog?

As you may know:

One Product  = One Product Owner = One Product Backlog

This is the same either in Scrum or Nexus. Both frameworks spin around the same product and the product has a single source of changes owned by one PO from where the team (or teams) pull work. To comply with the rules it's important to decide if the product is one product formed by multiple teams or each product is a separate product formed by one team. That is definitely something to sort out the sooner the better.

We're not really sure if this will going to work since the organization has just adopted agile.

One recommendations for nexus is to have previous scrum experience, which means also having knowledge of the agile performance of the teams. I'm not saying that forming a nexus right away and be successful is not possible but not having the basics of Scrum covered within the teams and organization could have a great undesirable impact. Apart form the nexus guide I'd recommend you the book The Nexus Framework for Scaling Scrum. It makes a good job guiding the process on forming a nexus through a case study.