Paak the caah in the backyaaad

So I have a super crush on a sort of coworker- and its totally innocent/inappropriate. He's 16 years older than me (but has the mentality of a 25 year old, but the maturity age-appropriate), and he's a contractor for my job. But he always lights up my day with his adorable accent, sarcasm, and on-point witty charm. Is it bad to have wildly inappropriate daydreams about him at work? We've both shown interest in each other, and I'm just waiting for him to make some sort of move. We enjoy each others' company, have had lunch together a few times, even made sexually-driven comments about 'naughty things' we'd do to each other (this is always said in jest, we do work for a government agency, we're not that bad). I have no intentions of anything serious, as I don't think he does either. But we have fun together- so whats the hold up? I want to hang out outside of work for once! See each other in civies, not badges and ties!