In simple terms optimization is making  the best choice  under given conditions. 
In mathematical terms this conditions are called constraints. 
so optimization is also called constained optimization. 

In some cases the problem what we are working on can be repersented by mathematical relationship . 

In some cases the  problem will not have a concrete mathematical representation. If we are able to represent the problem in mathematical terms the mathematical representation is called objective function. 

So when  we say we are trying to optimize the objective function ,The actual meaning is that we are trying to 
use the mathematical representation of our problem to find a best solution. 

For example , if we are flying an aircraft and we theoretically can take multiple routes to destination. But practially the duration of the flight is determined by the fuel tank capacity , the possibility of refuelling , the fatigue of the pilot etc. So as you can see the route to be taken depends on multiple constraints. 

so solving an optimization problem involves finding the contraints first and then achieve the best solution working under this condition. 

There are many popular optmization algorithms out there like , stochasitic gradient descent , QAOA (Quantum Approximate Optimization Algorithm) etc, 

The choice of algorithm depends on the problem which we are trying to solve. 
For example , 
The Stochastic Gradient Descent algorithm is used in Machile Learning problems using Neural Networks and The QAOA is used to problemswhich fall under the category Quadratic Unconstrained Binary Optimization(QuBO). 
The software which implements the optmization algorithm is called a Solver.

There are many solvers available both open source and paid. 

Google OR Tools contains solvers for different optimization problems, they can be imported in C++ , Java ,Python  etc and make things work. 

Pyomo is a python based optimization language which is also widely used. 

In quantum computing , generally the Optimization is a two step process , since the quantum circuits are expensive , a classical optimizer is run first to find the parameters to be passed to the quantum optimizer. 

The next post will show a working code using python for optimization , 

The below link provides a good example of how classical and quantum optimization algorithms work in tandem. 

https://qiskit.org/textbook/ch-applications/qaoa.html