Climate change optimism is justified – a complete transition from carbon to solar and wind power looks practical and affordable within a generation (published in The Guardian on 19 January 2017).

My first book on climate change was published 10 years ago. I looked at how responsible individuals could choose to run their lives to cut their carbon footprint.

Inevitably minimising your carbon footprint meant making some uncomfortable choices – stopping eating meat, for example, or giving up flying. Hair-shirtism, in short. In 2009, I advised individuals on how they could cut their carbon emissions by 10%.

I then disobeyed some of my own recommendations, flying to the US to visit a daughter at university, for example. Over the subsequent years, it increasingly seemed to me that changing western lifestyles in the way my first book suggested was going to be an impossible struggle.

Moreover, the world still had more than a billion people without an electricity supply. Any solution that didn’t give them access to power was unfair and unworkable. So I switched to writing about how we could substitute low-carbon energy sources for fossil fuels.

At first it seemed that renewable electricity would always be more expensive and solar power would languish unless it was heavily subsidised. Using alternative energy sources seemed difficult, expensive and inconvenient. I now think I was completely wrong.

In fact, optimism about successfully tackling climate change has never been more justified because 2016 was the year in which it finally became obvious that the world had the technology to solve the problem. Even as the political environment has darkened, the reasons have strengthened for believing that a complete transition to low-carbon energy is practical and affordable within one generation.

Andrew Simms is right that global temperatures will probably overshoot the 2C target. But that makes the urgency of an energy transition even clearer. Despair is no excuse for inaction.

Solar power costs around the world fell by an average of another 15% in 2016, meaning that electricity from the sun became the cheapest form of energy generation in places as diverse as Chile, parts of the Middle East and the south-west of the US. The world saw the lowest-ever auction price for solar electricity in Abu Dhabi.

China committed to adding about 40 gigawatts annually of solar panels in the next few years, more than half the new capacity installed across the entire world in 2016. India made similarly ambitious plans, meaning that these two countries will put more solar on the ground than the entire world did a couple of years ago.

As more photovoltaics are installed, costs fall in a predictable way, so that the drive towards using solar energy in Asia and elsewhere will feed through to lower prices across the whole world, including the UK. Even in gloomy Britain, the government now sees solar photovoltaics delivering electricity at less than 2% more than a new gas-fired power station in 2020, with costs continuing to fall thereafter. In sunny places around the world, solar may fall to less than half the price of fossil electricity within a decade.

Progress hasn’t been restricted to wealthier countries. In places without access to electricity grids, new businesses grew offering pay-as-you-go access to solar electricity.

Radically new approaches to solar technologies also saw major progress in 2016. The Dresden company Heliatek raised the money to build a new factory producing solar film that is printed on plastic backing and which can be simply pinned to the walls of buildings. Investors backed Oxford PV’s plans to start production of double-layer photovoltaic panels, using cheap and widely available “perovskite” materials originating in the university’s laboratories. Both companies demonstrated unexpected advances in the efficiency of their products at converting light to electricity.

Wind turbine costs declined as well, falling to levels not expected for at least another decade in the case of offshore installations in the North Sea. In a particularly good location off the Netherlands, one consortium promised to build a large wind farm for less than €55 (£48) a megawatt hour, less than half the UK government’s target for offshore wind in 2020. According to the investment bank Lazard, onshore wind in the best locations in the US is now two-thirds the cost of the cheapest new gas-fired power stations.

And progress wasn’t restricted to wind and solar photovoltaics. We saw the first anaerobic digestion plant added to an African grid, new geothermal heat collectors in the US, France, Kenya and several other countries, and the building of huge plants that collect the sun’s energy as heat, ensuring 24-hour availability of power in countries such as Morocco and Israel. Worldwide, more new renewable electricity capacity was added than all fossil fuels combined in 2015. The usually conservative International Energy Agency said that 60% of new electricity generation over the next five years will come from carbon-free sources.

Wind and solar, which are going to become the dominant sources of global energy, cannot provide reliable 24-hour power. But improvements in short-term storage have come at a dazzling rate. For sunny countries, this means solar power combined with batteries will offer cheap and always available electricity.

Tesla’s huge Gigafactory opened in Nevada, driving down battery costs further. Large storage systems were installed in hundreds of locations around the world to help electricity grids cope with variable wind and sun. As importantly, rapid advances in digital technology allowed electricity users large and small to respond to shortages of electricity by cutting their own use for short periods.

Meanwhile the Chevy Bolt, the world’s first mass-market, 200-mile electric car, reached showrooms in California. World electric car sales grew by more than 50%, achieving more than a quarter of all new registrations in Norway, the world’s most advanced market. Chinese EV manufacturers sold a third of a million vehicles and, very surprisingly, a senior VW executive said that electric cars may be as cheap as petrol vehicles within five years. And they are already much cheaper to run.

Even with extensive use of batteries, and continuing improvements in the ability of users to adjust electricity use, northern latitude countries will need more storage to cope with the short winter days and the possibility of weeks without wind. But the German company Electrochaea opened the first full-scale “power to gas” plant in the outskirts of Copenhagen in summer 2016. When electricity is abundant, and therefore very cheap, Electrochaea converts it into natural gas using microbes. This renewable gas can then be burned to generate power when electricity is scarce. In the autumn, Hungary signed an agreement to build an Electrochaea plant 10 times the scale of the Copenhagen trial. This is a technology with extraordinary potential for northern European countries.

Other new technologies showed how energy can be stored conveniently for months at a time, usually as liquid fuels similar to petrol or kerosene. Several airlines ordered aviation fuel from the many pioneers making it from sources such as wood chippings or carbon monoxide from steel mills. Much more research is needed on fuels but the outlines of how the world will wean itself off oil are clear. At its simplest, we will use the very cheap electricity produced by wind and solar to chemically convert simple carbon molecules back up to more complex and energy-carrying liquids that can substitute for petrochemicals.

As the world set new records for temperature, and the worrying effects of climate change grow more obvious by the week, the encouraging developments of 2016 provide some counterweight. For the first time we can hope that the momentum of low carbon growth is now unstoppable. In many cases around the globe, clean technology no longer needs any subsidy, a finding that seemed unachievable five years ago.

Even with sceptical politicians in charge of major democracies, the end of the era of fossil fuels is within view. And this is not just a renewables enthusiast talking; the CEO of Shell said that solar will eventually become the “dominant backbone” of the energy system.

The speed of change needs to increase but we can see how it might be possible to run an entirely carbon-free world, including giving abundant energy to people in the poorest countries.

I hope this is not an unduly complacent conclusion; we will still need to become far more efficient in our energy use and to push our politicians and business leaders to focus on sustainability. But we no longer have the excuse for inaction that it is too expensive or too difficult to wean ourselves off fossil fuel.

The Switch by Chris Goodall on a future based on solar power is available from the Guardian bookshop. He publishes a free weekly newsletter on clean energy around the world at www.carboncommentary.com.