Shaped Cost Function¶

API Reference¶

masa.common.ltl.ShapedCostFn ¶

ShapedCostFn(dfa: DFA, potential_fn: Callable[[int], float], gamma: float = 0.99)

Bases: DFACostFn

Potential-based shaped DFA cost for counterfactual experience.

This class implements a potential-based shaping term on top of the base DFA cost, intended for counterfactual computations where you explicitly pass a DFA state to DFACostFn.cost.

The shaped cost is:

\[ c'(q, \ell) = c(q, \ell) + \gamma \Phi(q') - \Phi(q), \]

where \(q'\) is the next automaton state after reading labels \(\ell\), \(c(q, \ell)\) is the base DFA cost, and \(\Phi\) is a user-provided potential function.

Important

This cost function is not intended to be used statefully.
reset and __call__ are disabled by design.

Creates a shaped DFA cost function.

Parameters:

Name	Type	Description	Default
`dfa`	`DFA`	DFA whose accepting states define the base cost.	required
`potential_fn`	`Callable[[int], float]`	Potential function \(\Phi(q)\) over DFA states.	required
`gamma`	`float`	Discount factor \(\gamma\) used in potential-based shaping.	`0.99`

Source code in masa/common/ltl.py

def __init__(self, dfa: DFA, potential_fn: Callable[[int], float], gamma: float = 0.99):
    r"""Creates a shaped DFA cost function.

    Args:
      dfa: DFA whose accepting states define the base cost.
      potential_fn: Potential function :math:`\Phi(q)` over DFA states.
      gamma: Discount factor :math:`\gamma` used in potential-based shaping.
    """
    super().__init__(dfa)
    self.potential_fn = potential_fn
    self._gamma = gamma

potential_fn `instance-attribute` ¶

potential_fn = potential_fn

_gamma `instance-attribute` ¶

_gamma = gamma

reset ¶

reset()

Disables resetting for shaped counterfactual cost.

Raises:

Type	Description
`RuntimeError`	Always raised. This object is intended for counterfactual calls to `DFACostFn.cost` only.

Source code in masa/common/ltl.py

def reset(self):
    """Disables resetting for shaped counterfactual cost.

    Raises:
      RuntimeError: Always raised. This object is intended for counterfactual
        calls to :meth:`DFACostFn.cost` only.
    """
    raise RuntimeError(
        "Shaped cost function is not supposed to be reset only used for counter factual experiences"
    )

cost ¶

cost(state: int, labels: Iterable[str]) -> float

Computes shaped cost from an explicit DFA state without mutation.

The shaped cost is:

\[ c(q,\ell) + \gamma \Phi(q') - \Phi(q), \]

where \(q' = \delta(q, \ell)\) is the DFA transition result.

Parameters:

Name	Type	Description	Default
`state`	`int`	DFA state \(q\) to evaluate from.	required
`labels`	`Iterable[str]`	Iterable of atomic proposition names \(\ell\) for the current step.	required

Returns:

Type	Description
`float`	Potential-based shaped cost.

Notes

This method does not change the wrapped DFA's internal state.

Warning

The implementation calls self.potential(state) for the final term, which assumes a method/attribute named potential exists. If you intended to use the provided callable, replace that with self.potential_fn(state).

Source code in masa/common/ltl.py

def cost(self, state: int, labels: Iterable[str]) -> float:
    r"""Computes shaped cost from an explicit DFA state without mutation.

    The shaped cost is:

    .. math::

       c(q,\ell) + \gamma \Phi(q') - \Phi(q),

    where :math:`q' = \delta(q, \ell)` is the DFA transition result.

    Args:
      state: DFA state :math:`q` to evaluate from.
      labels: Iterable of atomic proposition names :math:`\ell` for the
        current step.

    Returns:
      Potential-based shaped cost.

    Notes:
      This method does not change the wrapped DFA's internal state.

    Warning:
      The implementation calls ``self.potential(state)`` for the final term,
      which assumes a method/attribute named ``potential`` exists. If you
      intended to use the provided callable, replace that with
      ``self.potential_fn(state)``.
    """
    next_state = self.dfa.transition(state, labels)
    cost = float(next_state in self.dfa.accepting)
    potential_cost = 0.0 if next_state in self.dfa.accepting or state in self.dfa.accepting else \
      self._gamma * self.potential_fn(next_state) - self.potential_fn(state)
    return cost + potential_cost

call ¶

__call__()

Disables stateful calling for shaped cost.

Raises:

Type	Description
`RuntimeError`	Always raised. This object is intended for counterfactual calls to `DFACostFn.cost` only.

Source code in masa/common/ltl.py

def __call__(self):
    """Disables stateful calling for shaped cost.

    Raises:
      RuntimeError: Always raised. This object is intended for counterfactual
        calls to :meth:`DFACostFn.cost` only.
    """
    raise RuntimeError(
        "Shaped cost function is not supposed to be called only used for counter factual experiences"
    )

Shaped Cost Function¶

API Reference¶

masa.common.ltl.ShapedCostFn ¶

potential_fn instance-attribute ¶

_gamma instance-attribute ¶

reset ¶

cost ¶

__call__ ¶

potential_fn `instance-attribute` ¶

_gamma `instance-attribute` ¶

call ¶