The document legalization is of two types. There exists apostille and consular legalization. The needed type of legalization is determined by the state where the document will be presented. The document legalization makes your document legal overseas. Having your document legalized, you will be able to produce it to official authorities abroad. For example, if your document was issued in Greece, it is legal only on the territory of it. If you need to show it to the officials in another country, you should make it legal.
There are some exceptional cases when the documents do not need legalization and there exist some countries which signed bilateral treaty with Ukraine, abolishing the legalization. The legalization of a document is always provided on the territory of the state where it was issued.