Lithium battery charging-undervoltage
Yes, charging a Li-Ion cell at constant voltage without ever terminating the charge will likely destroy the cell. What will happen is that your battery will get (maybe slowly) to 4.0 V, and, if the voltage stays, the charging current will gradually decrease, and will decrease to zero.