'close by' is a compound of 'close' and 'by'. 'Close' (meaning 'near' or 'nearby') comes ultimately from Old French 'clos' (past participle of 'clore', 'to shut') and Latin 'claudere'/'clausus', while 'by' comes from Old English 'bi' meaning 'near, about'.

'close' entered Middle English from Old French 'clos' (from Latin 'clausus'), and the adverbial/prepositional use of 'by' comes from Old English 'bi'. Over time the two words were used together (e.g. Middle English usages like 'close by') and formed the stable phrase 'close by' in Modern English.

Initially related to the idea of being 'enclosed' or 'near/adjacent' through 'close' and the locative sense of 'by', the combined phrase came to be used specifically for 'near in distance' as in the current meaning 'not far away'.