This project addresses the question of decision-making for autonomous agents equipped with knowledge. In most real-world applications, such agents have to face a lot of challenges for taking optimal decisions: the environment is typically dynamic, uncertain, and only partially observable; it is described over an extremely large number of attributes; decision must be very fast. In order to design agents which are able to handle these problems, the Artificial Intelligence community has developed complementary approaches, in particular symbolic (logical) and numerical formalisms. Numerical formalisms (in particular, Markov networks, Bayesian networks, Markov Decision Processes and their derivatives) are typically suited for representing the stochastic effects of actions and the stochastic evolution of the environment, and can be learned and solved with various techniques. On the other hand, logical formalisms (e.g., attribute-value, relational, epistemic) are well suited for expressing hard constraints, norms, epistemic knowledge, goals, etc. In particular, logic is more declarative in essence, making it easier for humans to manipulate. Again, these formalisms come with various techniques for learning and reasoning. Our proposal stems from the observation that an agent placed in a real-world environment has typically access to some information through numerical models, and to some other in logical form. Then, a natural rationality requirement is that its decisions take all this information into account. Typically, a soccer robot needs a numerical model of its effectors (obtained by simulation or training), but also a logical model of the game rules. Another typical situation is medicine, where both numerical estimates of the efficiency of treatments or the exactness of analyses and logical expert knowledge are needed. So, the problem we will attack can be formulated as follows: design approaches for taking rational decisions when part of the information about the environment, actions, and rewards is given in numerical form, and part in logical form. A complete approach for handling this problem must take into account the following three subproblems: representation of the problem; optimal policy computation; reinforcement learning. We will attack these problems from the point of view of complexity-theory and algorithmics, hence identifying the complexity of problems, identifying tractable restrictions and designing efficient algorithms (both in terms of complexity and in practice). This is justified by the fact that most problems are already known to be computationally hard even when numerical and logical information are not considered together (e.g., in Partially Observable Markov Decision Processes - POMDPs). To that aim, we will build on existing factored representations of (PO)MDPs, especially by Dynamic Bayesian Networks and Probabilistic STRIPS Operators, which are based on propositional attributes. The focus on propositional logic rather than more expressive relational formalisms will ensure decidability of most problems, reasonable complexity, and possible reuse of very efficient software, e.g., for satisfiability. The proposed representations and algorithms will be illustrated on two large-scale applications. The first one consists in building occurrence maps of spatial processes, where the decisions to be taken are the locations to visit for getting information about the occurrence of the process. A real application is studied at INRA where the process to be mapped is the growth of invasive species. The second application concerns nonplaying characters in video games.